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Abstract 

This  report  outlines  a  framework  for  evaluating  ecosystem  restoration  benefits 
within  the  context  of  USACE  Civil  Works  planning  process.  An  emphasis  is 
placed  on  knowledge  gained  from  research  conducted  under  the  Environ¬ 
mental  Benefits  Analysis  (EBA)  Program  to  address  needs  for  a  science- 
based  assessment  strategy  that  moves  the  standard  of  practice  closer  to  the 
state-of-science  for  EBA.  Current  practice  can  be  improved  with  the  explicit 
use  of  conceptual  models,  establishment  of  clear  objectives  and  associated 
metrics,  better  predictive  tools,  quantification  of  uncertainty,  more 
structured  decision  methods,  and  adaptive  management.  The  report  is 
intended  to  assist  Corps  water  resources  planners  and  Project  Delivery 
Teams  (PDTs)  in  evaluating  ecosystem  restoration  projects,  programmatic 
considerations  and  general  advancements  in  restoration  science.  It  should 
also  assist  ERDC  researchers  and  program  managers  as  well  as  USACE 
water  resources  planners  in  aligning  EBA  products  to  meet  planning  needs. 


DISCLAIMER:  The  contents  of  this  report  are  not  to  be  used  for  advertising,  publication,  or  promotional  purposes. 
Citation  of  trade  names  does  not  constitute  an  official  endorsement  or  approval  of  the  use  of  such  commercial  products. 
All  product  names  and  trademarks  cited  are  the  property  of  their  respective  owners.  The  findings  of  this  report  are  not  to 
be  construed  as  an  official  Department  of  the  Army  position  unless  so  designated  by  other  authorized  documents. 

DESTROY  THIS  REPORT  WHEN  NO  LONGER  NEEDED.  DO  NOT  RETURN  IT  TO  THE  ORIGINATOR. 
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1  Introduction 

Purpose 

The  capability  to  identity,  quantify,  and  articulate  environmental  benefits  is 
central  to  the  formulation  of  sound  ecosystem  restoration  projects  and  is 
necessary  for  making  good  restoration  decisions.  This  technical  report 
presents  a  framework  for  assessing  benefits  expected  from  aquatic  eco¬ 
system  restoration  actions.  It  outlines  several  key  principles  necessary  for 
scientifically  sound  formulation  and  evaluation  of  restoration  alternatives 
and  is  intended  to  promote  good  practice.  The  framework  is  applicable 
within  the  Corps’  current  six-step  planning  process  and  is  deliberately 
flexible  so  it  can  be  adapted  to  differing  viewpoints  regarding  the  ways  in 
which  “benefits”  might  be  portrayed. 

Points  of  emphasis 

The  quality  of  an  environmental  benefits  assessment  (EBA)  is  contingent 
upon  a  number  of  factors.  This  report  presents  an  overarching  framework 
for  recognizing,  characterizing,  assessing,  and  evaluating  benefits  poten¬ 
tially  resulting  from  aquatic  ecosystem  restoration  actions,  and  focuses 
upon  the  following  factors  offering  opportunities  to  improve  practice: 

1.  Ecological  understanding.  The  relationships  among  the  hydrologic, 
geomorphic,  and  biologic  characteristics  of  ecosystems  are  the  foundation 
of  effective  ecosystem  restoration  and  benefits  assessment.  The  manipula¬ 
tion  of  these  relationships  through  engineering  practices  forms  the  basis 
for  the  Corps  of  Engineer’s  ecosystem  restoration  mission. 

2.  Objectives  and  metric  selection.  The  quality  of  assessments  and 
evaluations  rests  on  the  choice  of  metrics  used  to  compare  alternatives  and 
evaluate  the  results  of  investments.  Objectives  and  benefits  generally 
extend  beyond  habitat  improvements,  and  a  more  comprehensive 
accounting  of  restoration  benefits  is  desirable. 

3.  Forecasting.  Practitioners  should  embrace  modeling  and  related 
techniques  associated  with  forecasting  of  ecosystem  condition(s). 
Quantifying  expected  benefits  from  ecosystem  state  changes  requires  the 
development,  use,  and  interpretation  of  models. 
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4.  Uncertainty.  Good  EBA  practice  requires  the  recognition,  identification, 
characterization,  and  explicit  consideration  of  uncertainty  and  associated 
risks. 

5.  Reference  concepts.  Reference  concepts  are  important  to 
understanding  ecosystems  and  can  be  used  to  establish  a  “target”  for 
restoration.  Reference  conditions  can  also  be  used  to  index  ecosystem 
quality,  which  provides  a  means  for  quantifying  benefits. 

6.  Adaptive  management.  Adaptive  management  practices,  if  embraced 
and  exercised  during  a  project’s  planning  and  lifecycle  activities,  can 
significantly  improve  the  potential  for  realizing  restoration  benefits  and 
decrease  potential  for  making  regretted  decisions. 

7.  Documentation.  An  accurate  and  thorough,  but  concise,  accounting  of 
the  EBA  is  necessary  to  effectively  convey  information  to  decision  makers. 

Scope  and  limitations 

This  report  addresses  the  technical  aspects  of  an  environmental  benefits 
assessment  (EBA),  and  provides  a  framework  for  scientifically  sound 
practice.  This  framework  is  one  of  a  series  of  documents,  tools,  and  other 
resources  prepared  under  the  EBA  Research  Program.  The  reader  should 
use  this  document  as  an  entree  to  the  detailed  technical  information  and 
guidelines  incorporated  within  several  related  technical  notes.  The  latter 
are  referenced  throughout  this  report  and  will  be  available  online  at  the 
Ecosystem  Restoration  Gateway.1  This  document  describes  and/or 
otherwise  characterizes  the  state  of  science  or  field  and  practice  of 
ecosystem  restoration  and  should  in  no  way  be  perceived  as  an  expression 
of  Civil  Works  policy.  It  is  expected  that  readers  will  consult  formal  Civil 
Works  policy,  regulations,  and  guidance  for  details  regarding  Civil  Works 
policy  and  required/acceptable  practices  (U.S.  Army  Corps  of  Engineers 
(USACE)  1999a,  1999b,  1999c,  2000). 


1  http://cw-environment.usace.armv.mil/eba/index.cfm 
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2  Key  Strategies  and  Concepts 

Ecosystem  organization 

Assessing  the  environmental  benefits  that  arise  from  an  ecosystem 
restoration  project  requires,  among  other  things,  a  keen  understanding  of 
the  linkages  among  the  physical,  chemical,  and  biological  characteristics  of 
the  system.  The  effects  of  proposed  management  measures  on  the  eco¬ 
system  condition  over  time  and  the  value  society  places  upon  the  resultant 
changes  in  ecological  state  form  the  basis  for  benefits  quantification. 

Ecosystem  restoration  goals 

The  goal  of  the  Corps'  Civil  Works  ecosystem  restoration  activities  is  to 
“restore  significant  ecosystem  function,  structure,  and  dynamic  processes 
that  have  been  lost  or  degraded”  with  the  intent  of  partially  or  fully 
reestablishing  the  attributes  of  a  naturalistic,  functioning,  and  self- 
sustaining  system  (USACE  1999a).  This  stated  purpose  provides  context 
for  the  establishment  of  more  specific  planning  objectives  underpinning 
any  aquatic  restoration  effort  undertaken  by  the  Corps.  In  many  instances, 
a  full  return  to  pre-disturbance  conditions  may  not  be  feasible,  but  partial 
restoration  that  yields  significant  improvements  to  degraded  ecological 
resources  may  be  possible.  Actions  that  prevent  continued  degradation 
may  also  yield  significant  benefit,  even  if  no  “restoration”  occurs.  Thus,  the 
ability  to  assess  the  restoration  (in  part  or  whole)  of  the  structural  and 
functional  characteristics  of  ecosystems,  and  quantify  the  benefits 
resulting  from  those  improvements,  is  fundamental  to  the  EBA  process. 

Structure  and  function 

Ecosystem  restoration  is  accomplished  through  improvements  to  some,  and 
potentially  all,  of  the  structural  or  functional  conditions  or  characteristics  of 
the  system.  Ecosystem  form,  or  structure,  refers  to  both  the  composition 
of  the  ecosystem  and  to  its  physical  and  biological  organization  (National 
Research  Council  (NRC)  2005).  Structural  characteristics  vary  in  time  and 
space,  are  unique  to  each  system,  and  include,  for  example,  system 
morphology,  size  and  distribution  of  bed  sediments,  composition  of  the 
biotic  community,  and  the  system’s  hydrodynamic  regime. 
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Ecosystem  functions  are  the  physical,  chemical,  and  biological  processes1 
that  create  and  sustain  an  ecosystem  (Fischenich  2006).  Examples  of 
processes  that  sustain  biological  communities  include  primary  productivity, 
nutrient  cycling,  hydrological  processes,  erosion  and  sedimentation,  species 
interactions,  and  natural  disturbance  regimes.  Functions  are  largely 
responsible  for  the  “self-organizing”  and  dynamic  characteristics  of  eco¬ 
systems.  Structure  and  function  are  closely  linked  in  aquatic  ecosystems, 
such  that  change  to  one  likely  affects  the  other.  Reducing  sediment  yield  to  a 
stream,  for  example,  leads  to  channel  incision  and  widening.  Those 
functions  associated  with  hydrology  and  geomorphology  are  particularly 
relevant  to  Corps'  ecosystem  restoration  projects,  as  discussed  below. 

Hydrogeomorphic  process  focus 

Aquatic  ecosystem  restoration  has  historically  focused  on  biotic  habitat  and 
water  quality,  but  an  emergent  trend  has  emphasized  the  geomorphic 
structure,  function,  and  evolutionary  trajectory  of  systems  (Bennett  et  al. 
2009),  coupled  with  an  understanding  of  the  landscape  context  within 
which  ecohydrologic  processes  interact.  A  broad  “eco-hydrogeomorphic” 
understanding  is  necessary  to  frame  spatially  and  temporally  rigorous 
approaches  to  assessing  the  diversity,  variability,  and  complexity  of  aquatic 
ecosystems.  It  also  serves  as  a  foundation  for  identifying  and  characterizing 
benefits  in  a  fashion  consistent  with  the  Corps'  approach  to  ecosystem 
restoration.  Hydrologic  and  geomorphic  manipulations  are  the  primary 
management  measures  employed  by  the  Corps  for  aquatic  restoration,  and 
the  Corps  has  a  long  history  of  dealing  with  these  parameters.  Environ¬ 
mental  benefits  assessment  involves  translating  those 
hydrogeomorphic  manipulations  into  ecological  effects,  then 
assessing  those  effects  in  terms  of  objectives  that  reflect 
relevant  social  values. 

Restoration  Benefits 

Four  elements  are  required  to  effectively  quantify  aquatic  ecosystem 
restoration  benefits: 


1  The  terms  function  and  process  are  used  interchangeably  in  the  literature.  Herein,  the  term  function 
refers  to  a  broader  organization  of  interacting  processes.  For  example,  the  function  of  nutrient  cycling 
is  a  consequence  of  fixation,  mineralization,  nitrification,  denitrification,  and  other  chemical  and 
biological  processes. 
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1.  A  baseline  against  which  ecological  changes  can  be  compared  (this 
includes  ecological  changes  in  the  absence  of  any  project  and  is  the  “future 
without-project”  condition). 

2.  An  understanding  of  ecological  changes  likely  to  result  from  the 
restoration  action  (e.g.,  the  “future  with”  project  condition). 

3 .  A  timeline  (the  period  of  analysis) . 

4.  A  mechanism  for  recognizing  and  attributing  value  to  changes  in  ecological 
conditions  (i.e.  Is  the  condition  improved  or  made  worse?  By  how  much?). 

Standards  of  practice  have  emerged  for  the  first  three  elements,  and  some 
have  been  reasonably  standardized  for  application  within  the  Corps. 
However,  opportunities  for  improvement,  especially  where  criticisms  from 
the  science  community  have  been  leveled,  are  addressed  in  this  report. 
Several  mechanisms  for  associating  values  with  changes  in  ecosystem 
condition  exist,  but  no  single  approach  has  achieved  consensus  among 
scientists  or  the  community  of  practice. 

Values  and  the  origin  of  benefits 

Alterations  to  the  structure  and  function  of  an  ecosystem,  while  benefiting 
some  organisms  or  interests,  typically  come  at  the  expense  of  others.  For 
example,  the  removal  of  fine  silts  from  the  bed  of  a  stream  may  provide 
clean  gravel  substrate  benefitting  some  macroinvertebrate  species,  but  other 
members  of  the  invertebrate  community  may  be  adversely  impacted  by  this 
action.  Clearly,  the  benefits  of  any  ecosystem  restoration  action  depend 
upon  perspective,  and  the  perceived  benefits  of  restoration  reflect 
ecological,  socioeconomic,  cultural,  and  even  personal  values  (Clewell  and 
Aronson  2007).  Because  values  change  over  time  and  vary  among  stake¬ 
holders,  the  value  ascribed  to  restoration  activities  can  be  quite  subjective. 
Likewise,  the  proposal  of  restoration  actions  can  reveal  conflicts  and 
necessitate  consideration  of  trade-offs. 

Classes  of  benefits 

Fischenich  and  Payne  (in  preparation)  describe  three  classes  of  benefits  that 
utilize  different  metrics  and  reflect  alternative  viewpoints  of  the  implicit  and 
explicit  values  that  arise  from  healthy  ecosystems  (Figure  1).  These  concepts 
of  benefit  are  not  mutually  exclusive;  they  can  arise  from  the  same  restora¬ 
tion  action  but  reveal  different  views  of  the  technical,  cultural,  and  personal 
significance  of  ecosystems.  Any  or  all  viewpoints  can  be  employed  for 
benefits  assessment;  Table  1  provides  an  example  of  how  the  potential 
benefits  arising  from  a  stream  restoration  project  depend  upon  the  selected 
class  of  benefit. 
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Figure  1.  Classes  of  benefits  associated  with  ecosystem  restoration. 


Table  1.  Examples  of  benefits1  derived  from  a  stream  restoration  project  demonstrating  the  dependency  upon 

objectives  and  viewpoint. 


Social  Benefits  (goods  and  services) 

Objective-Based 

Benefits 

EPA/State 

DEQ 

USFWS/NMFS/ 
State  DFW 

Local  Government 

Local  Landowner 

Trout  Unlimited 

Water 

Temperatures 

Substrate 

Refugia 

Hydrologic 

Connectivity 

Sediment 

Processes 

Water 

Quality 

Standards 

Met 

Fish  Habitat 
Improvements 

Improved  Fish 

Community 

Composition 

Reduced  Erosion 

Improved  Water 
Quality 

Recreation 

Enhancements 

Increased 

Property  Taxes 

Increased 

Property  Values 

Aesthetic 

Enhancement 

Increased 

Native  Trout 
Numbers 

Improved 

Recreational 

Opportunities 

1  Note  that  the  ecosystem  benefits  approach  implicitly  encompasses  all  the  items  in  the  table  as  well  as  others  not 
specifically  identified  both  in  the  present  and  future. 


The  objective-based  approach  to  benefit  characterization  focuses  upon 
improvements  to  specific  ecological  attributes  of  an  ecosystem,  and  is 
presently  the  most  common  approach  among  the  three  classes.  Restoration 
consequences  can  also  be  expressed  in  terms  of  the  change  in  those  goods 
and  services  produced  by  ecosystems  that  are  valued  by  humans  (Mooney 
and  Ehrlich  1997).  The  provision  of  habitat  is  generally  regarded  as  an 
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ecosystem  service  and  has  long  been  used  for  assessing  restoration  within 
the  Corps,  but  conventional  application  extends  the  concept  to  a  much 
broader  suite  of  services.  The  third  viewpoint,  referred  to  as  ecosystem 
benefits,  avoids  the  need  to  segregate  or  aggregate  specific  benefits  and 
aims  to  implicitly  capture  the  full  suite  of  benefits  associated  with  a  healthy 
system.  This  approach  uses  a  suite  of  metrics  for  one  or  more  reference 
conditions  to  serve  as  a  benchmark  for  comparison,  assessment,  and 
evaluation.  Consensus  regarding  which  of  these  three  approaches  is  best  has 
not  been  achieved.  Rather,  selection  of  the  benefit  accounting  method 
depends  upon  how  that  information  is  going  to  be  utilized,  among  other 
factors. 

Need  for  benefits  quantification 

Results  that  emerge  from  a  benefits  assessment  are  fundamentally 
influenced  by  the  way  in  which  the  benefits  question  is  framed,  which  in 
turn  is  a  function  of  the  purpose  of  the  analysis.  Motivation  for  quantifying 
the  benefits  is  generally  one  or  more  of  the  following: 

1.  To  distinguish  between  different  actions,  projects,  or  programs. 

2.  To  characterize  return  on  investment  expected  from  a  restoration 
initiative. 

3.  To  prioritize  restoration  projects  given  finite  financial  resources. 

4.  To  maximize  desired  ecological  outcomes  resulting  from  every  dollar 
spent. 

5.  To  ensure  that  mitigation  requirements  are  met  or  to  calculate  banking 
credits. 

Evaluations  supporting  the  above  differ  in  that  alternative  comparison  and 
prioritization  may  only  require  that  the  relative  benefits  and  costs  be 
determined;1  detailed  or  comprehensive  quantification  may  be  unnecessary 
so  long  as  the  analyses  are  sufficient  to  allow  reasonable  comparisons.  The 
other  questions  demand  more  accurate  accounting  of  benefits  and 
consideration  of  other  factors  including  the  technical  and  institutional 
significance  of  the  targeted  ecosystem(s),  the  effectiveness  of  the  actions  at 
eliciting  the  desired  results,  the  resiliency  of  the  restored  system,  and  the 
fund  investment  efficiency. 


^Assuming  each  alternative  being  compared  satisfies  any  ecological  thresholds  critical  to  formulation. 
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Generic  framework  for  EBA 

As  is  the  case  with  most  planning  efforts,  the  activities  associated  with  EBA 
tend  to  be  iterative  in  nature,  and  vary  in  degree  and  scope  depending  upon 
specific  project  characteristics.  At  the  broadest  level,  EBA  consists  of  four 
general  phases  listed  below.  The  decision  phase  largely  involves  activities 
outside  benefits  “quantification,”  but  is  relevant  to  benefits  “assessment.” 

1.  A  qualitative  phase  involving  the  development  of  a  good  understanding  of 
the  ecosystem,  the  underlying  problems,  and  the  objectives  and 
constraints. 

2.  A  quantitative  phase  that  includes  forecasting  future  ecological  states, 
assigning  values  to  the  changes  in  ecological  condition  (through  metrics 
reflecting  change  in  ecological  function  or  structure),  and  assessing 
associated  costs  and  uncertainties. 

3.  The  decision  phase,  which  involves  an  assessment  of  the  outputs  relative  to 
planning  objectives,  constraints,  costs,  and  other  factors. 

4.  A  measurement  phase,  wherein  the  actual  benefits  are  determined  as  part 
of  a  monitoring  and  adaptive  management  program. 

Steps  in  the  EBA  process 

Benefits  analysis  involves  multiple  steps,  many  of  which  are  common  to  all 
assessments  and  some  that  depend  upon  the  specific  project  characteristics 
as  well  as  types  of  metric  and  valuation  techniques  that  are  applied.  Though 
the  process  is  presented  as  linear,  iterations  are  often  employed  in  practice. 
The  EBA  steps  outlined  below  are  not  necessarily  congruent  with  current 
practice,  but  can  be  implemented  within  the  Corps’  six-step  planning 
process:1 

Qualitative  phase 

1.  Ensure  a  sound  qualitative  understanding  of  the  ecosystem.  This  may 
require  the  development  of  a  conceptual  model2  representing  a  clear 


1  The  Corps’  planning  process  may  require  steps  not  specifically  listed  here,  or  may  express  a  different 
ordering. 

2  The  word  "model,"  as  used  throughout  this  report,  refers  to  a  simplified  representation  of  a  system  that 
allows  for  investigation  of  the  system  properties  or,  in  some  cases,  forecasting  likely  future  system 
states.  Many  kinds  of  models  are  applicable  to  EBA  (e.g.,  CEMs,  index-based  habitat  models, 
hydrologic  models,  etc.).  The  Corps  has  a  very  broad  definition  of  models,  including  "analytical  tools," 
which  might  consist  of  just  a  few  calculations. 
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understanding  of  the  causal  mechanisms  contributing  to  degradation  and 
possible  means  to  achieve  restoration  objectives. 

2.  Characterize  the  restoration  planning  and  design  objectives,  and 
specifically  define  the  spatial  and  temporal  scopes  of  the  analysis  and  the 
metrics  that  will  be  used  to  assess  the  future  ecological  states  and  degree  of 
success  attained.  This  step  may  utilize  information  from  a  reference 
condition  or  ecosystem. 

3.  Identify  the  restoration  measures  and  alternatives  to  be  evaluated  based 
upon  the  following  factors  (among  others): 

a)  How  the  various  actions  influence  the  ecosystem  processes  or 
condition  to  yield  desired  improvements. 

b)  Which  adaptive  management  opportunities  might  be  pursued  and 
how  they  may  affect  costs  and  outcomes. 

Quantitative  Phase 

4.  Compute  costs  (including  adverse  impacts)  for  each  alternative  for  the 
planning  period. 

5 .  F orecast  the  parameters  of  interest1  for  the  future  without-proj  ect  (FWOP) 
condition2 3  and  each  alternative.  This  can  be  a  complex  step,  potentially 
involving  different  models  and  analytical  tools  as  well  as  the  application  of 
professional  judgment. 

6.  Conduct  any  needed  sensitivity  and  uncertainty  analyses, 3  and  consider 
the  value  of  adaptive  management  by  assessing  the  costs  of  learning 
relative  to  the  benefits. 

7.  Apply  any  additional  valuation  approaches,  if  necessary  (e.g.  ecological 
production  functions,  monetization  of  outputs,  application  of  significance 
modifiers,  etc.)  to  assess  the  outputs  (i.e.  benefits)  for  each  alternative. 

This  process  may  require  combining  several  metrics  and  involve  additional 
modeling.  Outputs  are  typically  based  upon  the  annualized  differences  in 
benefit  between  each  alternative  and  the  FWOP  condition. 


1  Examples  of  ecological  forecast  parameters  include  temperature,  velocity,  depth,  population  size,  etc. 
These  often  differ  from  the  output  (benefit)  metrics,  which  are  addressed  in  step  7. 

2  Within  the  Corps  planning  process,  forecasting  the  FWOP  condition  typically  precedes  the  development 
of  alternative  plans  and  their  evaluation.  In  this  generic  representation  of  the  process,  the  semi- 
quantitative  forecasting  of  FWOP  is  assumed  as  part  of  step  1,  whereas  the  quantification  of  the  FWOP 
and  the  with-project  alternatives  is  shown  concurrently  because  the  tools,  metrics,  and  approach  are 
the  same  for  all  conditions  and  scenarios  that  are  evaluated. 

3  Note  that  uncertainty  analysis  occurs  at  several  steps  in  the  process  and  tends  also  to  be  iterative. 
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Decision  phase1 

8.  Conduct  cost-effectiveness  (CE)  evaluation  and  incremental  cost  analyses 
(ICA)  for  the  remaining  alternatives  using  the  outputs  from  Step  7, 
including  the  analyses  believed  necessary  to  adequately  understand 
sensitivities  to  uncertainty  in  forecasted  ecological  conditions. 

9.  Make  needed  comparisons  among  the  alternatives,  applying  appropriate 
decision  criteria/factors  (key  thresholds,  project  constraints,  etc.). 

10.  Select  the  alternative  that  reasonably  and  most  cost-effectively  satisfies 
planning  objectives  and  prepare  the  documentation2 3  needed  to  effectively 
communicate  the  process  and  results. 

Confirmation  phase 

11.  Monitor  and  adaptively  manage  the  project  as  required  to  assess  success, 
maximize  achievement  of  project  objectives,  and  improve  EBAfor  future 
projects  through  knowledge  gained  from  monitoring  and  assessment. 

Change  as  a  basis  for  benefit  quantification 

To  provide  meaningful  input  to  decision-makers,  it  is  important  that 
computed  benefits  and  costs  reasonably  reflect  important  changes  that 
occur  to  the  ecosystem  as  a  consequence  of  the  restoration  actions. 
Ecosystems  are  not  static;  their  condition  changes  over  time  in  response  to 
both  natural  and  anthropocentric  influences  and  change  can  be  expected 
even  absent  any  intervention.  Consequently,  the  basis  for  evaluating  project 
benefits  is  the  change  over  time  in  the  “state”  of  the  ecosystem,  as  evidenced 
by  key  metrics  reflecting  the  quantity  and/or  quality  of  ecosystem  resources 
or  services.  Figure  2  illustrates  the  impact  of  change  when  analyzing  the 
benefits  of  restoration  alternatives. 

The  baseline  is  referred  to  as  the  future  without-project  (FWOP)  condition, 
and  is  represented  by  the  projected  system  conditions  over  the  planning 


1  Actions  in  the  decision  phase  are  outside  benefits  “quantification,”  but  are  presented  here  for  context 
in  benefits  “assessment.”  See  the  Institute  of  Water  Resources  (IWR)  Planning  Suite 
http://www.pmcl.com/iwrplan/  for  decision  support. 

2  Documentation  should  be  continuous  throughout  the  process;  this  step  is  aimed  at  compiling  that 
documentation  and  adding  necessary  tables,  figures,  etc.  for  effective  communication. 

3  Metrics  used  to  describe  the  condition  depend  upon  the  benefit  characterization  strategy,  and  might 
include  ecological  state  variables,  an  expression  of  the  service  production,  or  an  index  relative  to  a 
reference  condition. 
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Figure  2.  Schematic  representation  of  benefit  curves  for  restoration  alternatives.  The  shaded 
area  represents  the  net  benefits  (i.e.  "output")  for  Alternative  3.  Note  that  the  y-axis  units 
could  be  habitat  units,  population  size,  a  reference-based  index,  or  some  other  ecological 
benefit  metric,  depending  on  the  study  design. 

timeframe  (typically  50  years1)  without  the  proposed  restoration  project.2 
The  incremental  benefit  afforded  by  any  alternative  is  the  area  between  the 
benefit  curve  for  that  alternative  and  the  curve  for  the  FWOP  condition,  and 
is  referred  to  as  the  "output"  in  Corps  planning.  The  net  benefit  is  typically 
annualized  by  dividing  the  area  between  the  curves  by  the  number  of  years. 
In  Figure  2,  Alternative  3  has  a  net  benefit  of  10,400  units  (the  area  between 
Alternative  3  and  FWOP  curves)  and  an  average  annual  benefit  of  208  units 
(10,400  -r  50  years). 

Dealing  with  non-monetary  benefits 

Traditional  water  resources  development  employs  cost-benefit  analysis  for 
project  evaluation  (i.e.  the  benefits  on  the  y-axis  of  the  above  figure  would 
be  expressed  in  dollars).  There  are  many  challenges  to  the  assignment  of 
monetary  values  to  ecosystems  or  ecosystem  outputs  (NRC  2005,  Hussen 
2004,  Randall  1991,  Freeman  1993).  In  any  case,  current  policy  mandates 
the  use  of  non-monetary  metrics  for  ecosystem  restoration  projects.  Thus,  a 
central  need  for  ecosystem  restoration  planning  involves  the  development 


1  Longer  periods  may  be  employed  depending  upon  ecological  response.  Depending  on  the  variation  in 
the  ecological  outputs,  it  may  be  necessary  to  assess  conditions  at  weekly,  monthly,  annual,  or  decadal 
time-steps  to  assure  an  accurate  reflection  of  the  overall  benefits. 

2  Other  actions  likely  to  occur  during  the  timeframe  are  included  in  the  FWOP,  but  the  action  being 
considered  is  excluded  from  the  analysis. 
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and  use  of  non-monetary  metrics  that  represent  the  predicted  change  in 
outputs  of  interest  to  decision-makers. 


The  definition  and  measurement  of  restoration  outputs  should  follow 
logically  from  planning  objectives.  Based  on  Figure  l,  three  broad  types  of 
planning  objectives  that  may  be  relevant  include  the  promotion  of  l)  natural 
processes  and  dynamic  properties  that  drive  ecosystem  self-design  (e.g., 
hydrology  and  geomorphology),  2)  desired  ecological  resources  (e.g., 
wildlife  habitat),  and  3)  restoration  of  the  ecosystem  to  a  desired  reference 
condition  (e.g.  minimally  disturbed  condition).  Each  of  the  objective 
categories  can  have  a  set  of  associated  metrics,  and  these  typically  vary 
among  projects.  Table  2  provides  an  example  of  different  metrics  that  might 
apply  to  the  restoration  of  a  stream  degraded  by  urbanization. 


Table  2.  Examples  of  metrics  potentially  applicable  to  an  urban  stream  restoration  project. 


Ecological  Objective-Based 

Service-Based 

Reference- 

Structural 

Functional 

Other  Process 

Resources 

Services 

Based 

Channel  Cross 

Bed  Level  Change 

Stream  bank 

Habitat 

Downstream 

RCI  (i.e.  Ratio 

Section  Area 

Erosion  Rates 

Quantity 

Pollutant 

of  Project  to 

Floodplain 

Inundation 

Concentration 

Reference  for 

Baseflow  Pool 

Reproductive 

Groundwater 

Gradient 

Attribute  Set) 

Depth 

Frequency 

Success 

Recharge  Rate 

Flood  Wave 

Water 

Biological  Oxygen 

Riparian 

Attenuation 

Temperature 

Demand1 

Species 

Regeneration* 

Property  Values 

Number  of  Fish 
per  Mile 

Macroinvertebrate 

Diversity1 

WRT  Distance 
From  Stream 

Index  of  Biotic 
Integrity1 

1  Indicates  an  indirect  measure. 


Regardless  of  the  management  objective  strategy,  projecting  future  eco¬ 
logical  states  using  relevant  metrics  is  necessary.  This  is  usually  accomp¬ 
lished  through  the  application  of  ecological  production  functions.  Ecological 
production  functions  are  mathematical  expressions  that  estimate  the  effects 
of  changes  in  the  structure,  function,  and  dynamics  of  an  ecosystem  on 
outputs  that  are  directly  relevant  and  useful  to  decision  makers.  In  the 
context  of  Corps’  ecosystem  restoration,  several  functions  may  be  necessary 
(see  Figure  3).  Each  “function”  can  be  regarded  as  a  model,  and  different 
functions  are  generally  required  for  each  project,  or  perhaps  more  generally, 
for  each  type  of  project. 
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Figure  3.  Ecological  production  functions  are  used  to  assess  system  response  overtime  in  terms  of  metrics 
that  relate  to  the  project  objectives  (adapted  from  U.S.  Environmental  Protection  Agency  (USEPA)  (2009)). 


The  selection  of  appropriate  metrics  for  non-monetary  benefits  is  difficult 
and  contentious,  and  there  are  no  generally  accepted  standards,  although 
criteria  for  assessing  metric  quality  have  been  proposed  (McKay  et.  al. 
2012).  When  the  units  for  metrics  are  not  dollars,  other  decision  support 
methods  may  be  needed  to  evaluate  alternatives.  Techniques  such  as  cost- 
effectiveness  (CE)  evaluations  and  incremental  cost  assessments  (ICA) 
provide  a  way  of  comparing  alternatives  when  using  nonmonetary  metrics, 
Other  decision  analysis  methods  (e.g.  Multi-Criteria  Decision  Analysis 
(MCDA))  can  be  helpful  when  multiple  metrics  are  required  as  well 
(Suedel  et.  al.  2012).  These  tools  are  introduced  in  the  following  sections. 

Economic  assessment  (CE/ICA) 

CE/ICA  is  a  form  of  efficiency  analysis  that  serves  to  refine  and  illustrate 
tradeoffs  among  a  set  of  alternatives  for  which  the  benefits  are  expressed 
in  a  non-monetary  metric.  Cost  effectiveness  (CE)  analysis  compares  the 
costs  of  each  project  plan  against  its  non-monetary  measure  of  output  and 
screens  out  plans  that  are  not  cost-effective  from  further  consideration. 
The  incremental  cost  (IC)  analysis  identifies  incremental  costs  per  unit 
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output  gained  by  moving  from  one  cost-effective  plan  to  the  next  higher- 
output  cost-effective  plan.  The  combined  use  of  CE/ICA  may  not  identify  a 
single  “best”  plan,  but  it  does  eliminate  those  plans  that  are  demonstrably 
inferior  to  others.  The  approach  is  widely  used  on  federal  water  resource 
development  projects,  and  tools  exist  to  help  in  its  implementation 
(Institute  for  Water  Resources  (IWR)20io). 

Addressing  multiple  benefits 

The  "benefits"  shown  on  the  Y-Axis  of  Figure  2  can  be  based  on  a  single 
metric  or  several  different  metrics  that  can  be  individually  referred  to  as 
benefits.  For  example,  a  proposed  restoration  alternative  might  improve 
stream,  riparian,  and  wetland  habitats.  These  benefits  can  be  combined 
into  a  single  proxy  metric  for  CE/ICA  evaluation,  or  may  be  treated 
separately  in  the  evaluation  process.  The  former  approach  has  the  benefit 
of  simplifying  comparisons  (although  the  supporting  analyses  may  be 
complex  and  problems  can  arise  when  combining  dissimilar  metrics), 
whereas  the  latter  approach  provides  flexibility  in  dealing  with  trade-offs. 
A  separate  accounting  of  benefits  is  generally  necessary  in  cases  where  a 
project  contributes  to  multiple  benefit  categories  (national  economic 
development  (NED),  environmental  quality  (EQ),  regional  economic 
development  (RED),  and  other  social  effects  (OSE)).1 

While  it  is  desirable  to  "fully"  account  for  project  benefits,  comprehensive 
valuation  of  aquatic  ecosystems  by  summing  specific  benefits  is  typically 
impractical  and  potentially  misleading.  This  is  different  from  composite 
valuation  of  whole  ecosystems  because  it  involves  quantifying  and  valuing 
specific  processes,  services,  and  characteristics  with  interdependent  yet 
uncertain  behaviors,  then  combining  these  through  mathematical 
manipulation.  This  does  not  mean  that  ecosystem  valuation  cannot  be 
accomplished,  simply  that  summation  of  specific  benefits  to  arrive  at  a 
total  value  should  be  approached  with  caution.  Important  concepts  to  keep 
in  mind  when  addressing  multiple  benefits  include  the  following: 

•  Account  only  for  those  benefits  that  can  be  reasonably  identified  and 
substantiated. 

•  Focus  on  benefits  that  clearly  affect  decision-making  and  are  relevant 
to  the  project  objectives;  a  core  group  of  critical  benefits  is  preferable 
to  an  exhaustive  list. 


1  Consult  current  policies  regarding  the  handling  of  multiple  accounts. 
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•  Establish  interdependencies  between  benefits. 

•  Avoid  double-counting  of  benefits. 

•  Identify  trade-offs  among  conflicting  benefits. 


Addressing  trade-offs 

At  several  points  in  the  EBA  process,  decisions  are  needed  regarding 
alternative  approaches  that  present  various  trade-offs.  Table  3  summarizes 
methods  that  can  help  ensure  structure,  order,  transparency,  and  repeat¬ 
ability  in  the  decision-making  process  (Guinto  2008).  These  and  other 
techniques  (see  Belton  and  Stewart  2002)  can  be  useful  for  identifying 
relevant  criteria,  determining  objectives,  selecting  metrics,  formulating 
alternatives,  illustrating  and  evaluating  trade-offs,  resolving  conflicts  and 
consensus-building,  recommending  a  plan,  and  developing  adaptive 
management  strategies. 


Table  3.  Common  decision  analytic  (DA)  approaches  (after  Guinto  2008). 


Method 

Description 

Considerations 

Ordinal  ranking 

Criteria  and/or  alternatives 
are  simply  ranked  in  order 
of  preference 

Decision  makers  know  preference  only; 
they  do  not  know  ‘by  how  much’  one 
alternative  is  preferred  over  another.  A 
concern  is  that  cardinal  weights  for 
aggregation  may  still  be  needed. 

Rating 

Users  rate  criteria  and/or 
alternatives  on  a  scale  of 
0-100.  Likert  scales  can 
also  be  used. 

This  method  is  simple  and  transparent 
and  does  not  constrain  the  decision 
makers’  responses. 

Point  allocation/ 
Fixed  Point  Scoring 

Decision  makers  allocate  a 
fixed  number  of  points 
such  as  100,  10,  etc. 

The  point  allocation  method  is  simple 
and  transparent  and  requires  users  to 
make  trade-offs  by  budgeting  100  points 
among  attributes. 

Paired  comparison 
(e.g.,  analytic 
hierarchy  process) 

Requires  the  decision 
maker  to  consider  each 
criterion  against  every 
other  criterion  in  pairs. 

If  the  number  of  decision  criteria  is  not 
limited,  the  task  of  making  comparisons 
becomes  overwhelming  and  can  lead  to 
inconsistent  and/or  useless  results. 

EBA  and  the  Corps'  planning  process 

This  report  provides  a  technical  framework  for  incorporating  the  tasks 
necessary  to  evaluate  environmental  benefits.  Table  4  diagrams  the 
relationship  between  the  phases  of  EBA  and  the  Corps’  iterative  six-step 
planning  process.  The  EBA  framework  presented  in  this  report  consists  of 
four  discrete,  and  also  iterative,  phases  that  emphasize  the  more  unique  or 
critical  aspects  of  the  planning  process  as  related  to  environmental 
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Table  4.  Six-step  planning  process  and  EBA  phases. 


Planning  Process 

EBA  Activities 

1-Specify  Problems 
and  Opportunities 

Develop  an  understanding  of  the  ecosystem  to  be  restored  including  the 
history,  landscape  setting,  source  of  problem,  etc. 

Qualitative  Phase 

Prepare  a  conceptual  ecological  model  including  the  drivers,  stressors, 
ecological  responses,  and  indicators  (or  equivalent  factors  if  using  a 
different  type  of  OEM).  The  OEM  may  include  other  factors  if  needed. 

Identify  the  spatial  and  temporal  scales  for  evaluation  and  key 
uncertainties.  Determine  the  potential  for  adaptive  management. 

Identify  key  uncertainties  and  preliminary  reference  conditions. 

State  the  project  objectives.  Choose  among  objective-,  service-  or  whole 
ecosystem-based  approaches  for  characterizing  benefits. 

2--lnventory  and 

Forecast  Conditions 

Establish  metrics  and  performance  measures  for  the  project  objectives 
and/or  targeted  ecological  parameters. 

Quantitative  Phase 

Evaluate  existing  forecasting  models  and  select  the  preferred 
model  or  modify  or  develop  the  model  needed  to  forecast  conditions. 

Forecast  the  FWOP  and  alternative  scenarios  (if  required),  including 
appropriate  sensitivity  and  uncertainty  analyses. 

3-Formulate 

Alternative  Plans 

Assess  potential  for  adaptive  management  and  include  appropriate 
considerations  in  alternative  formulation. 

Revisit  and  adjust  the  CEM,  objectives,  and  constraints  as  needed. 

4--Evaluate  Effects  of 
Alternative  Plans 

Forecast  the  ecological  conditions  for  each  alternative  plan. 

Assess  forecasting  data  to  compute  net  changes  to  ecosystem  quantity 
and/or  quality,  annualize  outputs;  make  any  necessary  changes  to  metrics 
to  capture  values  in  terms  of  output  metrics. 

Quantify  and  assess  uncertainties  for  each  plan. 

5--Compare  Alternative 
Plans 

Conduct  cost-effectiveness  and  incremental  cost  analyses. 

Decision 

Phase 

Apply  other  decision  criteria  based  on  thresholds,  constraints,  reference 
conditions,  stakeholder  criteria,  etc.,  and  identify  the  recommended  plan. 

Document  the  process  and  information  helpful  to  decision  makers. 

6-Select 

Recommended  Plan 

Monitoring  and  adaptive  management  implementation  phase  is  required  to 
substantiate  and  optimize  benefits. 

AM  Phase 

restoration  and,  in  particular,  EBA.  Table  5  identifies  key  products 
developed  under  the  EBA  Research  Program  and  references  them  to  the 
appropriate  phase/step.  The  referenced  products  are  available  on  the 
Ecosystem  Restoration  Gateway  at:  http://cw-environment.usace.armv.mil/cwenv.cfm. 
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Table  5.  Key  EBA  documents  cross  referenced  to  relevant  EBA  phase  /  planning  step. 


EBA  Phase 

Qualitative 

Quantitative 

Decision 

M&AM 

Planning  Step 

1-Specify 
Problems  and 
Opportunities 

2-Inventory 
and  Forecast 

Conditions 

3-Formulate 

Alternative 

Plans 

4-Evaluate 

Effects  of 

Alternative 

Plans 

5-Compare 

Alternative 

Plans 

6-Select 

Recommended 

Plan 

N/A 

Reference  Documents 

Fischenich  -  Science-Based 

Framework  for  Environmental 
Benefits  Assessment  (ie.  this 
report) 

X 

X 

X 

X 

X 

X 

X 

Fischenich  -  The  Application  of 
Conceptual  Models  to  Ecosystem 
Restoration  Projects  (Fischenich 
2008) 

X 

X 

X 

McKay  -  Ecosystem  Restoration 
Objectives  and  Metrics  (McKay  et 
al.  2012) 

X 

X 

X 

Linkov  -  Metric  Selection  for 
Ecosystem  Restoration  (Linkov,  in 
preparation) 

X 

X 

X 

Swannack-  Ecological  Modeling 
Guide  for  Ecosystem  Restoration 
and  Management  (Swannack  et  al. 
2012) 

X 

X 

Suedel  -  Application  of  Risk 
Management  and  Uncertainty 
Concepts  and  Methods  for  Eco¬ 
system  Restoration:  Principles  and 
Best  Practice  (Suedel  et  al.  2012) 

X 

X 

X 

Convertino  -  An  Illustrative  Case 
Study  of  the  Application  of 
Uncertainty  Concepts  and  Methods 
for  Ecosystem  Restoration 
(Convertino  et  al.  2012) 

X 

X 

X 

Miller  -  Reference  Concepts  in 
Ecosystem  Restoration  and 
Environmental  Benefits  Analysis 
(EBA):  Principles  and  Practices 
(Miller  etal.  2012) 

X 

X 

Pruitt  -  The  Use  of  Reference 
Ecosystems  as  a  Basis  for  Assessing 
Restoration  Benefits  (Pruitt  et  al. 
2012) 

X 

X 

X 

Fischenich  -  The  Application  of 
Adaptive  Management  to  Ecosystem 
Restoration  Projects  (Fischenich  et 
al.  2012) 

X 

X 

X 

X 

Conyngham  -  Guidance  on 
Monitoring  Requirements  and 
Principles  of  Practice  in  USACE 
Ecosystem  Restoration  Efforts  (In 
preparation) 

X 
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Phase  l  of  the  EBA,  the  qualitative  phase,  is  consistent  with  step  l  of  the 
planning  process  ( the  specification  of  problems  and  opportunities).  This 
phase  contains  tasks  predominantly  related  to  understanding  the  ecosystem 
and  the  study  objectives.  Specific  activities  in  this  phase  include  developing 
a  conceptual  ecologic  model  (CEM),  identifying  the  appropriate  temporal 
and  spatial  scales  for  system  evaluation,  giving  initial  consideration  to  an 
adaptive  management  approach,  and  identifying  key  uncertainties.  If  a 
reference  concept/ecosystem  is  to  be  employed,  classification  and  prelimi¬ 
nary  identification  of  the  reference  is  undertaken.  The  conclusion  of  this 
phase  is  the  development  of  planning  objectives  for  the  restoration  effort. 

Phase  2  of  the  EBA,  the  quantitative  phase,  involves  steps  2  through  4  of  the 
planning  process.  This  phase  begins  with  establishing  metrics  and  perfor¬ 
mance  criteria  that  relate  to  the  planning  objectives.  The  initial  quantitative 
activity  is  consistent  with  planning  step  2  ( inventory  and  forecast  condi¬ 
tions),  and  involves  an  array  of  activities  related  to  evaluating  the  ecological 
state  of  the  system,  including  selecting  or  developing  appropriate  evaluation 
method(s)  and  forecasting  techniques/models,  forecasting  the  future 
without-project  condition,  and  preparing  scenarios  (if  required).  Ecological 
attributes  used  for  reference  systems  are  identified  and  quantified  if  a 
reference  approach  is  employed. 

Planning  step  3  (formulate  alternative  plans)  includes  the  development  of 
management  measures  for  alternative  plans/projects.  This  activity  falls 
within  the  quantitative  phase  of  EBA,  but  is  not  strictly  a  part  of  the  EBA 
process.  However,  alternative  formulation  includes  a  deliberate  revisit  of 
the  study's  objectives  and  constraints,  a  refreshing  of  the  CEM  to  accommo¬ 
date  responses  to  potential  alternatives,  and  consideration  of  the  potential 
for  and  implications  of  adaptive  management. 

Planning  step  4  (evaluate  effects  of  alternative  plans)  is  the  focus  of  the 
second  half  of  the  quantitative  phase  of  EBA.  This  includes  evaluating  the 
effects  associated  with  specific  alternatives,  refining  or  combining  metrics  to 
most  effectively  capture  the  estimated  benefits,  quantifying  uncertainties, 
and  computing  the  outputs  from  alternatives  over  the  study  period.  For  a 
whole-ecosystem  approach,  the  selected  system  attributes  for  the  with- 
project  conditions  are  compared  and  indexed  to  the  reference  condition. 

Phase  3  of  the  EBA  framework,  the  decision  phase,  corresponds  to  steps  5 
and  6  of  the  planning  process  (compare  alternative  plans  and  recommend 
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an  alternative  plan).  Phase  3  includes  the  cost-effectiveness  and  incre¬ 
mental  cost  analyses,  subjecting  the  plans  to  the  various  decision  criteria, 
and  selecting  the  recommended  plan.  Documentation  developed  during  the 
process  is  refined  and  additional  information  is  presented  that  is  needed  to 
effectively  communicate  the  benefits  and  decision  processes  used  to  arrive 
at  the  selected  alternative. 

The  fourth  EBA  phase  includes  monitoring  and  implementing  adaptive 
management  to  help  secure  the  projected  benefits  and  confirm  the  actual 
benefits  realized  by  the  project.  This  phase  marks  the  final  set  of  activities 
needed  to  complete  an  EBA.  Monitoring  and  adaptive  management  do  not 
have  a  correlate  in  the  Corps’  planning  process,  as  they  are  typically 
regarded  as  operations  activities.  However,  knowledge  gained  from 
monitoring  and  adaptive  management  does  influence  future  planning,  the 
implementation  of  adaptive  actions  influence  benefits  realized,  and  the 
ability  to  adaptively  manage  affects  planning  decisions. 
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3  Overarching  Technical  Issues 

While  many  factors  are  associated  with  successfully  conducting  an 
environmental  benefits  assessment,  opportunities  for  improvements  in 
practice  are  especially  obtainable  for  six  topics  discussed  in  this  chapter: 
understanding  the  ecosystem,  metric  selection  and  combination,  modeling 
and  forecasting,  addressing  uncertainty,  adaptive  management,  and 
documenting  and  communicating  decisions. 

Understanding  the  ecosystem 

An  EBA  cannot  advance  in  the  absence  of  a  clear  understanding  of  the 
problems  leading  to  ecological  degradation,  the  relationship  between 
proposed  remediation  measures  and  ecological  response,  and  the 
relationships  among  ecological  conditions  and  those  factors  that  are 
valued  by  society.  This  section  addresses  the  role  of  reference  concepts, 
conceptual  ecological  models,  and  thresholds  and  response  trajectories  in 
developing  the  above  understanding. 

Reference  systems 

The  use  of  reference  approaches  in  environmental  assessments  is  becoming 
more  prevalent  (Bowman  and  Somers  2005).  Reference  approaches 
generally  involve  comparing  characteristics  of  a  ‘project’  site  to  those  found 
in  a  minimally  impacted  ‘reference’  site  or  to  a  suite  of  reference  sites 
representing  some  range  of  conditions.  Restoration  success  hinges  upon  a 
sound  understanding  of  the  ecosystem  being  restored  and  a  guiding  vision 
for  the  restoration  project.  A  reference  ecosystem  provides  a  clear  model  of 
the  intended  outcome  as  well  as  a  means  for  benchmarking  success.  Thus, 
reference  systems  can  be  used  to  help  develop  a  conceptual  model,  identify 
restoration  objectives,  establish  success  metrics,  and  serve  as  a  baseline  for 
monitoring  efforts,  among  other  things. 

Reference  models  have  traditionally  been  based  upon  conditions  prior  to 
human  disturbances.  Such  a  historical  reference  is  often  unavailable, 
unachievable,  or  even  undesirable  and  alternative  formulations  of  reference 
condition  have  evolved  (Stoddard  et  al.  2006).  A  reference  might  be  an 
actual  site  or  sites;  for  example,  a  nearby  ecosystem  in  a  similar  setting. 
Alternatively,  the  reference  might  reflect  a  statistical  model  based  upon 


ER  DC/EL  TR-13-4 


21 


several  sites  or  even  a  "conceptual  ecosystem"  if  actual  sites  are  not 
available.  Miller  et  al.  (2012)  describe  approaches  for  developing  and 
classifying  reference  systems,  and  Table  8  presented  later  in  this  document 
summarizes  the  considerations  regarding  use  of  references  in  EBA. 

Role  of  conceptual  models 

Conceptual  ecosystem  models  (CEMs)  are  descriptions  of  the  general 
functional  relationships  among  essential  components  of  a  system.  They 
tell  the  story  of  “how  the  system  works”  with  respect  to  key  processes  and 
attributes  and,  in  the  case  of  ecosystem  restoration,  how  the  proposed 
alternatives  aim  to  alter  the  processes  or  attributes  of  the  restoration  site 
to  benefit  the  system  (Fischenich  2008).  The  development  of  a  CEM  is 
recommended  as  a  first  step  in  the  planning  process,  as  it  provides  a  key 
link  between  early  planning  (e.g.,  an  effective  statement  of  problem,  need, 
opportunity,  and  constraint)  and  later  evaluation  and  implementation. 
Fischenich  and  Barnes  (in  preparation)  present  a  case  study  detailing  the 
CEM  development  process,  and  other  case  study  examples  are  available  at 
the  Ecosystem  Restoration  Gateway. 

Conceptual  models  can  be  invaluable  in  supporting  EBA  because  they 
promote  understanding  of  ecosystem  processes,  help  formulate  goals  and 
objectives,  metrics,  and  alternatives,  and  play  an  important  role  in  selecting 
evaluation  methods  and  associated  performance  metrics  for  monitoring 
adaptive  management.  Detailed  guidance  on  the  development  of  CEMs  can 
be  found  in  Fischenich  (2008),  and  a  tool  to  assist  the  preparation  of 
conceptual  models  is  publicly  available  (Dalyander  and  Fischenich  2010). 
CEMs  provide  many  benefits,  but  the  following  four  are  of  special 
significance  to  EBA. 

1.  A  CEM  documents,  in  brief,  the  scientific  rationale  for  engineered 
modifications  and  the  externalities  (such  as  climate  change  and  sea-level 
rise)  that  might  affect  the  degree  to  which  modifications  are  likely  to  elicit 
desired  ecosystem  responses. 

2.  A  CEM  includes  the  essential  ecosystem  structure,  functions,  and 
processes  to  be  measured,  as  well  as  those  that  may  be  unquantifiable  but 
need  to  be  considered  in  the  overall  evaluation.  Thus,  the  CEM 
summarizes  the  rationale  for  metrics  used  in  EBA. 

3.  During  CEM  development,  uncertainties  affecting  the  project's  success  are 
identified,  discussed,  and  vetted.  This  initial  assessment  of  key 
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assumptions  (hypotheses)  sets  the  stage  for  subsequent  evaluation  and 
documentation  of  uncertainty  in  the  ERA. 

4.  When  constructed  collaboratively,  CEMs  facilitate  communication,  foster 
consensus,  and  capture  the  collective  expertise  of  participating  scientists, 
agencies,  and  the  public,  thus  making  the  overall  assessment  of  benefits 
and  final  plan  selection  more  likely  to  gain  support  from  decision  makers 
and  the  public. 

Trajectories  and  thresholds 

An  understanding  of  potential  ecological  trajectories  is  necessary  for 
developing  effective  alternatives,  evaluating  benefits,  and  implementing 
adaptive  management  measures.  The  concept  of  ecological  trajectory  with 
respect  to  both  the  impairment  and  recovery  of  a  restored  ecosystem  is 
represented  in  Figure  4.  In  the  figure,  degradation  has  moved  an  ecosystem 
site  from  its  “original  condition”  to  a  degraded  state  that  is  the  present  focus 
of  restoration.  A  wide  range  of  potential  alternative  states  are  shown  along 
the  right  margin  of  the  figure.  The  least  complex  (most  degraded)  of  these 
indicates  the  forecasted  state  of  the  site  with  no  restoration  (i.e.  the  FWOP). 
The  somewhat  better  future  states  of  the  site  indicate  a  range  that  might  be 
derived  from  two  alternative  plans  that  consist  of  either  removing  the 
stressor  leading  to  degradation  or  undertaking  other  restorative  measures 
(or  both).  The  ecological  trajectory  initially  pushed  upward  by  restoration 
measures  might  not  be  linear,  and  the  outcome  can  be  uncertain,  requiring 
scenario  analyses. 

Ecosystem  ecology  has  moved  in  recent  decades  away  from  linear,  single 
equilibrium  state  models  of  ecosystem  behavior  to  an  increased  recognition 
of  the  importance  of  dynamism  and  the  investigation  of  thresholds  and 
multiple  meta-stable  states  (Hobbs  and  Suding  2010).  This  recognition  both 
complicates  forecasting  of  restoration  outcomes  and  potentially  improves 
decision-making,  given  that  assessments  of  restoration  benefits  are  likely  to 
be  more  realistic.  Probability-based  assessments  that  explicitly  account  for 
key  ecological  thresholds  and  alternative  end  states  provide  a  sound  basis 
for  alternative  comparisons  and  the  application  of  adaptive  management. 
Additional  guidance  on  the  identification  and  application  of  threshold  and 
trajectory  concepts  can  be  found  in  Conyngham  and  Fischenich  (in 
preparation). 
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Figure  4.  Various  ecological  trajectories  and  alternative  future  states  of  an  impaired  site. 


Metric  selection  and  combination 

As  applied  in  this  report,  the  term  metric  refers  generically  to  a  quantifiable 
system  property  used  to  measure  something,  typically  progress  toward  a 
goal  or  objective  (McKay  et  al.  2010,  2012).  Metrics  are  used  in  the  EBA 
process  for  forecasting  ecosystem  response  to  proposed  restoration 
alternatives,  to  inform  decision  making,  and  to  report  outcomes  of 
restoration.  The  fundamental  requirements  for  establishing  technically 
sound  metrics  can  apply  to  any  of  four  uses  in  ecosystem  restoration: 

1.  Ecosystem  state  variables  refer  to  the  measurable  physical,  chemical,  and 
biological  properties  of  the  system  that  serve  as  targets  for  the  restoration 
action;  they  generally  should  be  the  parameters  being  manipulated  by  the 
engineering  action  and  forecasted  over  the  future  with-  and  without- 
project  (for  example,  peak  summer  temperature,  mean  substrate  size, 
frequency  of  floodplain  inundation,  population  size,  etc.)  condition. 

2.  Output  metrics  are  the  primary  basis  for  assessing  benefits;  they  are  used 
for  cost-effectiveness  and  incremental  cost  analyses  in  Corps  ecosystem 
restoration  planning  (for  example,  average  annual  habitat  units  based  on  a 
combination  of  the  above  factors).  Output  metrics  can  be  the  same  as 
ecosystem  state  variables. 
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3.  Decision  factors  refer  to  thresholds  in  output  value,  acceptable  uncertainty 
or  risk,  stakeholder  preferences,  and  other  criteria  used  in  conjunction 
with  CE/ICA  for  decision  making.  These  are  not  strictly  necessary  for  EBA. 

4.  Performance  measures  are  the  monitoring  parameters  used  to  assess 
project  success  or  for  triggering  adaptive  management  actions  (Fischenich 
et  al.  2012),  and  either  directly  (preferably)  or  indirectly  reflect  the  above 
metrics. 

Universal  metric  sets 

Given  the  diversity  of  aquatic  ecosystem  types,  widely  varying  geographic 
factors,  and  an  incredible  range  of  possible  ecosystem  functions,  goods,  and 
services,  it  is  no  surprise  that  a  single  metric  set  applicable  to  all  aquatic 
ecosystem  restoration  projects  has  failed  to  emerge.  Good  metrics  must 
evolve  from  project  objectives  and  most  projects  have  multiple  objectives 
that  vary  widely.  Therefore,  no  single  objective  set  applicable  to  all  restora¬ 
tion  projects  can  be  specific  enough  for  decision-making  purposes. 

Metric  development  and  evaluation 

Good  metrics  evolve  from  and  can  be  directly  mapped  to  specific  project 
objectives.  Metrics  that  directly  measure  an  objective,  and  that  are 
recognized,  documented,  and  widely  used,  are  generally  preferred. 
However,  it  is  sometimes  necessary  to  use  indirect  metrics  or  to  develop 
metrics  that  might  not  have  been  previously  applied  and  documented. 
Candidate  metrics  and  metric  sets  can  be  evaluated  based  on  whether  they 
adequately  address  project  objectives  and  the  extent  to  which  they  possess 
the  following  qualities  of  good  metrics  (McKay  et  al.  2010): 

•  Relevance.  The  metrics  address  project  objectives  and  priorities,  are 
scientifically  valid  at  appropriate  temporal  and  spatial  scales,  and  are 
appropriate  to  project  decisions. 

•  Ambiguity.  The  metrics  clearly  measure  the  consequences  of  different 
alternatives,  reveal  direction  of  response,  and  have  minimal  natural 
and  computational  uncertainty. 

•  Comprehensiveness.  Taken  together,  the  metric  set  addresses  the 
full  suite  of  project  objectives  and  project  consequences. 

•  Directness.  The  metrics  directly  quantify  project  outcomes.  If 
aggregate  scores  or  indices  are  used,  these  are  constructed  and 
documented  in  a  way  that  supports  direct  and  clear  interpretation  in 
terms  of  project  effects. 
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•  Operability.  The  metrics  can  be  developed,  forecast,  and  potentially 
monitored  within  the  time,  labor,  and  budget  limits  for  the  project. 

•  Understandability.  The  metrics  can  be  communicated  in  plain 
language  to  decision-makers  and  the  public. 


Example:  Developing  useful  objectives  and  metrics. 

The  following  example  objectives  and  metrics  are  presented  for  a  hypothetical  stream  restoration 
project,  and  are  characterized  as  poor,  fair,  or  good.  An  explanation  of  the  characterization  and 
needed  improvement  is  provided. 

Objective  /  Metric 

Discussion 

Restore  degraded  sections  of  River  “X”  by 
converting  class  D  stream  reaches  to  class  C. 
Metric:  Linear  feet  of  class  C  stream  in  study 
area. 

Poor.  Objective  should  not  specify  an  alternative; 
there  may  be  other  ways  of  achieving 
restoration.  Metric  only  addresses  habitat 
quantity;  should  also  account  for  quality  (unless 
class  is  quality  based). 

Improve  habitat  diversity  to  support  native 
aquatic  species  in  the  lower  10  miles  of  River 
“X.” 

Metric:  Population  size  for  species  “Y.” 

Poor.  Objective  is  vague  and  could  be  improved. 
Metric  not  sufficiently  comprehensive,  may  be 
difficult  to  quantify  and  may  be  a  poor  indicator. 

Improve  fish  passage  for  native  species  at  river 
mile  (RM)  3.4  and  RM  5.2  of  River  “X.” 

Metric:  Benthic  Macroinvertebrate  Index  (BMI). 

Poor.  Objective  is  OK.  Metric  is  not  clearly 
relevant  and  at  best  would  be  a  weak  indicator. 
Use  a  metric  that  quantifies  connectivity  or  is  a 
reasonable  measure  offish  passage. 

Restore  habitat  in  lower  10  miles  of  River  “X”  to 
more  closely  match  Minimally  Disturbed 

Condition  (MDC)  based  on  reference  systems. 
Metric:  Average  Annual  Habitat  Units  (AAHUs) 
based  on  depth,  velocity  and  substrate. 

Fair.  Objective  OK;  could  be  improved  with  more 
specificity.  Metric  not  well  understood, 
especially  by  public,  and  is  not  easily  scalable 
outside  the  immediate  project. 

Restore  connectivity  from  mile  0  to  10  for  native 
aquatic  organisms  as  evidenced  by  successful 
cutthroat  trout  reproduction  above  RM  3.4. 
Metrics:  Passage  efficiency  (%  passing)  and 
habitat  quality  (AAHUs  based  on  spawning 
needs). 

Fair.  Objective  could  be  improved  by  adding  a 
timeframe  (eg.  “...trout  reproduction  by  year  5 
above...”).  Metrics  may  be  difficult  to  combine, 
but  otherwise  OK.  Reproductive  success  (if 
predictable)  would  be  a  more  direct  metric. 

Restore  habitat  in  lower  10  miles  of  River  “X”  to 
more  closely  match  conditions  for  Minimally 
Disturbed  reference  systems. 

Metric:  Reference  Condition  Index  (RCI)  based 
on  a  suite  of  characteristics  and  processes. 

Fair.  Objective  could  be  improved  with  more 
specificity.  Metric  may  not  be  well  understood, 
especially  by  public,  but  is  scalable  outside  the 
project  and  potentially  across  ecosystem  types. 

Lower  temperatures  in  side  channel  habitats  to 
provide  summer  refugia  for  the  endangered  “Z.” 
Metric:  3-day  maximum  summer  temperature  in 
side  channels. 

Good.  Objective  is  clear.  Metric  is  relevant, 
direct,  operable,  understandable,  and 
predictable. 

Restore  floodplain  connectivity  in  the  study 
reach  to  permit  annual  inundation  of  at  least  3 
days  between  March  15  and  April  30. 

Metric:  Acres  of  floodplain  meeting  inundation 
criteria. 

Good.  Objective  and  metric  meet  all  criteria. 
Additional  objectives  and  metrics  may  apply 
unless  floodplain  connectivity  is  the  only 
degraded  process. 
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Metric  combinations  and  comparisons 

Although  multiple  metrics  may  be  used  during  planning,  the  EBA  itself 
should  use  only  a  subset  of  metrics,  ideally  few  in  number,  that  have  clear 
relevance  to  decisions  about  plan  selection  and  project  justification. 
However,  because  the  performance  of  a  metric  across  the  range  of  alterna¬ 
tives  cannot  be  wholly  anticipated  at  the  outset,  there  is  a  danger  in 
reducing  the  number  of  metrics  for  the  EBA  prematurely.  One  option  is  to 
develop  a  variety  of  metrics  with  different  scales  and  expected  response 
thresholds  and  then  filter  and  refine  these  after  initial  forecasting  has  been 
conducted  in  order  to  reduce  the  number  of  metrics,  improve  their 
resolution,  and  ensure  that  benefits  and  impacts  of  alternatives  are  not 
overlooked. 

The  ability  to  compare  metrics  that  measure  diverse  objectives  is  important 
to  ecosystem  restoration  decision  making,  particularly  for  more  complex 
projects  involving  multiple  objectives.  Techniques  facilitating  metric 
comparison  and  combination  have  been  well-studied  and  may  be  coarsely 
divided  into  four  major  categories:  l)  narrative  description,  2)  arithmetic 
combination,  3)  multicriteria  decision  analysis  (MCDA),  and  4)  interde¬ 
pendent  combination.  Each  of  these  techniques  has  associated  advantages 
and  disadvantages  that  are  discussed  in  more  detail  by  McKay  et  al.  (2012) 
and  Convertino  et  al.  (in  preparation). 

Modeling  and  forecasting 

Modeling  is  a  necessary  and  central  activity  of  an  EBA.  Modeling  is  essential 
for  analyzing  environmental  systems;  it  provides  the  means  to  forecast 
ecosystem  response  to  restoration  and  convert  those  forecasted  conditions 
into  quantified  benefit  estimates.  The  modeling  process  helps  organize 
thoughts  and  contributes  to  an  improved  understanding  of  ecosystems. 
However,  models  do  have  limitations.  Because  environmental  models  are 
simplified  representations  of  complex  systems,  they  are  often  built  using 
assumptions  regarding  the  unknown  components  in  the  model.  They  don't 
yield  "the  answer;"  rather,  they  provide  information  to  support  decision¬ 
making.  The  usefulness  of  a  model  hinges  on  understanding  whether  the 
data  and  assumptions  used  by  the  model  are  sufficient  to  inform  decisions. 
Model  utility  sometimes  can  be  judged  (and  improved)  by  post-project 
monitoring  that  reveals  the  accuracy  of  the  model  predictions. 
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Types  of  models 

Several  different  types  of  models  can  be  used  to  determine  environmental 
benefits.  These  range  from  simple  empirical  relations  describing  the 
expected  habitat  preferences  of  species  to  complex,  multi-dimensional, 
dynamic  models  of  material  flow  (water,  sediment,  etc.),  to  agent-based  or 
spatially-explicit  models  that  address  large-scale  dynamics  (e.g.,  Foran  et  al. 
2012,  Guisan  and  Zimmermann  2000).  In  general,  the  models  used  for 
environmental  benefits  fall  into  six  basic  classes  (conceptual,  analytical, 
index-based,  processed-based/ simulation,  statistical,  and  spatial),  each  with 
their  strengths  and  weaknesses  as  well  as  potential  applicability  within  the 
EBA  process  (see  Table  6). 


Table  6.  Description  of  model  types  used  for  EBA  (from  Swannack  et.  al.  2012). 


Model  Type 

General  Use 

Example 

Analytical 

Systems  where  solution  to  closed  form  equations 
represent  benefits 

Population  growth,  Lotka- 
Volterra  models 

Conceptual 

Diagramming  relationships  among  components, 
organizing  information,  determining  data  needs 

CEMCAT,  see  Fischenich 
(2008)  for  more  examples 

Index 

Determining  habitat  quality  across  a  landscape, 
relates  species  presence  to  environmental  variables 

Habitat  Suitability  Index 
models,  Hydro-Geomorphic 
Method 

Simulation 

Modeling  dynamics  of  complex  systems  that  have 
multiple  factors  interacting  across  scales,  often  have 
spatial  components 

Agent-based  models,  ADH- 
CASM,  ELAM,  system 
dynamic  models 

Statistical 

Analysis  of  datasets  to  determine  distributional 
properties  of  the  data. 

AN  OVA,  goodness-of-fit, 
regression,  t-test,  other 
empirical  models 

Spatial 

Projects  where  particular  spatial  attributes  are 
important  can  be  incorporated  into  simulation  models 

GIS-based  models, 

Ecological  Dynamics 
Simulation  Model 

Forecasting  ecological  conditions 

A  wide  variety  of  ecological  forecasting  tools  are  available  to  support  the 
project  planning  process.  These  range  from  relatively  simple  empirical 
relations  describing  the  expected  habitat  preferences  of  species  (or  guilds 
of  species)  to  complex,  dynamic  models  of  water  movement,  sediment,  and 
other  material  fluxes,  to  behavior-based  models  of  individual  organisms 
(agent-based  models)  and  spatially-explicit  tool(s)  that  address  habitat 
and  landscape  mosaics  (Foran  et  al.  2012).  The  choice  of  a  tool  or  set  of 
tools  should  be  based  on  the  specific  ecological  objectives  for  the  project, 
although  several  other  factors  not  directly  related  to  a  formal,  technical 
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evaluation  of  tool  capability  or  suitability  invariably  come  into  play  (e.g. 
project  size,  data  availability,  funding,  duration,  etc.). 

Regardless  of  the  particular  tool  set,  forecasting  ecological  conditions 
involves  quantifying  the  future  states  of  key  variables  describing  an  eco¬ 
system's  structure,  processes,  dynamics,  or  some  other  relevant  charac¬ 
teristic.  The  forecast  is  made  for  both  the  future  without-project  (baseline) 
condition  as  well  as  for  each  alternative  being  evaluated.  Given  the  inherent 
uncertainty  in  such  forecasts,  it  is  beneficial  if  the  tools  being  applied  can  be 
used  to  help  quantify  this  uncertainty  either  directly  in  the  form  of 
probability  distributions,  or  indirectly  through  sensitivity  analyses,  scenario 
analyses,  or  other  appropriate  means.  The  results  of  these  analyses  may 
serve  directly  as  the  basis  for  evaluating  benefits  in  the  case  of  ecologically 
focused  objectives,  or  may  require  further  analysis  (often  involving  addi¬ 
tional  tools)  to  convert  the  results  into  meaningful  outputs  that  capture  the 
appropriate  values  (see  Section  titled  “Metric  combination”). 

Computing  outputs 

Index  models  have  been  commonly  used  as  the  basis  for  computing 
outputs  for  Corps  restoration  projects.  These  models  typically  use  species- 
habitat,  community  habitat,  biotic  integrity,  and  functional  capacity 
indexes  to  reflect  relative  quality  of  a  system  anchored  in  some  optimal 
condition  of  maximum  quality  and  varying  downward  toward  zero  as 
conditions  change  from  optimum.  Quality  indices  and  geographical  area 
are  typically  “integrated”  by  multiplying  unit  area  (e.g.,  l  acre)  by  the  unit 
quality  index  and  summing  the  multiples.  One  example  of  the  product  of 
this  multiplication  is  the  habitat  unit  of  Habitat  Evaluation  Procedure 
(HEP)  (U.S.  Fish  and  Wildlife  Service  (USFWS)  1981),  which  can  be 
compared  directly  to  other  habitat  units  of  different  spatial  quantities  and 
quality  index  values  only  in  ideal  circumstances.  Alternatively,  Index  of 
Biological  Integrity  (IBI)  (Karr  1981)  and  some  other  multi-metric  index 
models  scale  over  a  broader  range  and  are  intended  to  reflect  biological 
condition  relative  to  desired  reference  conditions  independent  of  stream 
length  or  wetland  area. 

Including  benefits  that  extend  beyond  the  traditional  habitat  perspective 
implies  that  other  model  types  will  likely  be  required.  Depending  upon  the 
objectives,  certain  forms  of  statistical  models,  population  models,  or  other 
model  types  can  be  useful  for  quantifying  outputs.  In  some  circumstances, 
existing  models  will  be  available  to  quantify  the  parameters  of  interest; 
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often,  the  model  will  need  to  be  developed  or  adapted  from  an  existing 
tool.  Various  analytical  models  might  also  be  constructed  to  integrate 
benefits  arising  from  different  sources  of  ecological  change.  Additional 
guidance  is  provided  in  Swannack  et  al.  (2012). 


Example:  Model  Outputs 

The  eastern  oyster,  Crassostrea  virginica,  also  known  as  the  American  oyster,  Atlantic  oyster,  or  the  Virginia 
oyster,  is  a  species  of  oyster  that  is  native  to  the  eastern  seaboard  of  North  America. 

Crassostrea  virginica  may  be  the  target  of  restoration  efforts  because  of  a  direct  interest  in  the  species, 
because  they  are  an  indicator  of  estuarine  health,  or  because  they  are  an  important  component  of  the 
coastal  ecosystem  and  affect  other  species.  Modeling  for  oyster  restoration  efforts  can  take  many  forms 
depending  on  the  goals  and  objectives,  availability  of  data  and  other  resources,  and  other  study  needs. 
Although  the  ecological  requirements  for  oysters  are  the  same,  the  outputs  differ  depending  on  the  modeling 
approach,  as  shown  in  the  following  examples. 

Habitat  Suitability  Modeling. 

One  approach  to  evaluating  restoration  alternatives  is  through  the  use  of  a  Habitat  Suitability  Index  (HSj) 
model.  Oysters  have  preferences  for  specific  environmental  factors  such  as  salinity,  temperature,  depth,  and 
substrate.  Professional  judgment  or  statistical  analyses  can  be  used  to  identify  ranges  of  values  for  these 
variables  that  can  be  expressed  in  the  form  of  an  index,  ranging  from  0  (extremely  poor)  to  1  (excellent).  For 
example,  salinities  lower  than  5  ppt  or  higher  than  40  ppt  may  be  lethal  to  oysters,  and  would  be  given  an 
index  value  of  0.  Salinities  from  10  to  30  ppt  may  be  ideal,  and  are  scored  1.0,  with  the  ranges  from  5-10 
and  30-40  ppt  expressed  as  a  linear  function  between  0  and  1.  The  resultant  “suitability  curve”  is  used  to 
assign  index  scores  based  on  any  predicted  or  measured  value  of  the  variable.  Suitability  curves  for  the  other 
model  variables  are  likewise  developed,  and  scores  for  each  variable  are  assigned  and  input  into  an  equation 
that  combines  the  variables  to  yield  a  single  Suitability  Index  (SI)  value  (eg.  SI  =  (Vi  *  V2  *  V3  *  V4)A1/4 ,  where 
the  subscripts  refer  to  each  variable.  The  SI  value  is  multiplied  by  the  available  habitat  (usually  in  acres)  to 
yield  a  Habitat  Unit  (HU).  Values  for  HU  are  forecasted  for  several  points  in  future  time,  and  the  sum  of  the  HU 
values  is  divided  by  the  total  time  to  yield  an  Average  Annual  Habitat  Unit  (AAHU),  which  is  the  model  output. 
Values  for  AAHU  can  be  compared  among  alternatives  and  against  a  future  condition  without  any  restoration. 
Although  AAHUs  can  be  thought  of  in  terms  of  a  “quality-weighted”  expression  of  available  habitat,  the  units 
have  little  meaning  to  resource  professionals  and  even  less  to  the  public.  Comparing  outputs  for  different 
projects  is  difficult  at  best  and  more  typically  meaningless  because  the  construct  of  the  model  equations  is 
open  to  interpretation  and  can  be  manipulated  to  yield  either  higher  or  lower  values.  The  use  of  an  average 
value  for  habitat  units  might  disguise  important  trends,  and  assessing  the  relative  contributions  of  quantity 
and  quality  can  be  difficult.  Nevertheless,  this  is  a  common  modeling  approach  and  is  usually  within  the 
means  of  a  study  team. 

Population  Modeling. 

Another  approach  to  evaluating  oyster  restoration  alternatives  is  through  the  use  of  models  to  assess 
population  features  such  as  size  and  age  structure.  Population  characteristics  are  a  function  of  the  same 
habitat  conditions  as  described  above,  along  with  other  processes  such  as  spawning,  recruitment,  growth, 
and  mortality.  Population  models  can  be  statistical-  or  process-based,  and  often  rely  upon  the  use  of  other 
models.  For  example,  a  hydrodynamic  model  might  be  used  to  forecast  salinity  ranges  within  the  study  area 
for  each  of  the  proposed  alternatives  and  for  the  FWOP,  with  output  from  the  hydrodynamic  model  input  to  a 
statistical  model  for  growth  rate.  Outputs  from  population  models  typically  focus  on  relevant  population 
parameters  such  as  population  size,  age  structure,  biomass/unit  area,  etc.,  and  these  can  be  compared 
among  alternatives. 

The  outputs  from  population  models  are  usually  more  easily  understood  by  and  relevant  to  decision  makers 
and  the  public  than  are  outputs  from  habitat  suitability  models.  Evaluating  an  investment  that  would  result  in 
a  20%  increase  in  AAHUs  is  likely  more  difficult  and  abstract  than  an  investment  that  would  yield  a  40% 
increase  in  sustainable  harvest,  for  example.  The  added  fidelity  and  acuity  offered  by  population  modeling 
comes  at  a  price,  however.  The  models  can  be  much  more  complex,  require  more  data,  and  have  higher 
associated  uncertainties,  among  other  limitations. 


ER  DC/EL  TR-13-4 


30 


Other  Ecosystem  Models. 

Depending  on  the  project  objectives  and  associated  metrics,  other  modeling  approaches  might  be  required. 
For  example,  if  objectives  either  directly  or  indirectly  relate  to  the  effects  of  oysters  in  improving  water  quality, 
a  model  of  seston  uptake  (or  other  similar  factor)  would  be  applied  based  upon  assumed  oyster  densities, 
distribution,  water  column  mixing,  and  seston  uptake  rates,  among  other  factors.  The  outputs  would  ideally 
be  related  to  water  quality  (eg.  percent  seston  reduction,  turbidity,  etc.,)  in  these  situations.  If  water  quality  is 
an  intermediate  concern,  the  model  outputs  might  relate  to  the  other  objectives.  For  example,  oyster  uptake 
of  nutrients  and  resulting  reduction  in  turbidity  might  be  shown  to  improve  submerged  aquatic  vegetation 
(SAV)  distribution  or  health.  In  that  case,  several  models  might  be  employed  but  the  model  outputs  would 
relate  to  the  SAV  characteristics  of  interest.  Similar  concepts  could  apply  when  evaluating  oysters  as  a  food 
source  for  marine  fisheries;  the  modeling  may  involve  assessing  oyster  populations  or  habitat  as  an 
intermediate  step,  but  the  outputs  should  be  related  to  the  metrics  for  the  overall  project  objectives. 


Model  selection,  adaptation,  and  development 

Determining  the  “best”  models  to  use  for  evaluating  restoration  of  aquatic 
ecosystems  is  situational,  depending  on  a  number  of  factors  including:  the 
specific  processes  or  conditions  needing  evaluation,  required  accuracy, 
available  resources  (expertise,  time,  funding),  needed  data,  and  institutional 
acceptability.  Selecting  from  available  models  can  be  daunting  because  of 
the  large  number  and  variety  of  existing  models.  In  many  cases,  the 
“correct”  model  does  not  exist,  and  a  model  must  be  developed  or  adapted 
to  meet  the  needs  of  the  specific  project  and  circumstances. 

Swannack  et  al.  (2012)  provide  generalized  guidance  regarding  the  steps 
that  constitute  good  practice  for  evaluating  models  for  technical  efficacy. 
Candidate  models  may  be  evaluated  using  the  following  five  steps: 

1.  Evaluate  correspondence  between  model  results  and  expected  patterns  of 
model  behavior. 

2.  Examine  correspondence  between  model  projections  and  data  from  real 
system  (model  validation). 

3.  Adjust  empirical  parameters  or  model  coefficients  to  match  a  known 
behavior  (model  calibration). 

4.  Determine  levels  of  uncertainty  associated  with  model  projections. 

5.  Identify  data  gaps  and  research  needs  that  may  not  have  been  obvious 
during  conceptual  model  development. 


In  the  event  that  an  existing  model  does  not  meet  the  needs  of  a  given  study, 
Swannack  et  al.  (2012)  also  detail  the  process  of  model  adaptation  and 
development.  The  basic  approach  is  to  1)  develop  a  conceptual  model  of  the 
specific  cause-effect  relationships  among  important  components  of  the 
system,  2)  quantify  these  relationships  based  on  analysis  of  the  best  infor- 
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mation  possible,  which  can  include  scientific  data  or  expert  opinion, 

3)  evaluate  the  usefulness  of  the  model  in  terms  of  its  ability  to  simulate 
system  behavior,  and  4)  apply  the  model  to  address  needed  questions.  In 
practice,  model  development  does  not  proceed  linearly  from  the  conceptual 
model  to  model  application;  rather  it  iterates  through  a  series  of 
intermediate  models  of  increasing  complexity. 

Addressing  uncertainty 

Corps  policy  requires  that  uncertainty  in  water  resource  planning  be  evalu¬ 
ated  and  communicated.  Methods  for  assessing  uncertainty  in  ecosystem 
restoration  projects  continue  to  evolve,  but  include  sensitivity  analyses, 
scenario  planning,  and  parametric  uncertainty  analysis.  These  and  other 
means  of  identifying,  quantifying,  evaluating,  and  otherwise  considering 
uncertainties  as  part  of  the  planning  process  provide  important  information 
that  assists  decision-making.  Important  considerations  include  the 
following: 

•  Uncertainty  should  be  identified  early  in  planning  and  efforts  should 
be  made  to  reduce  the  causes  of  analytical  uncertainty  as  resources 
permit. 

•  Residual  sources  of  uncertainty  should  be  classified  as  to  type,  analyzed 
and  documented,  and  then  addressed  iteratively  throughout  the 
planning  process,  from  CEM  development  through  benefits  estimation, 
plan  selection,  and  adaptive  management  plan  development. 

•  Uncertainty  should  be  quantified  where  possible,  and  confidence 
intervals  or  probability  distributions  used  as  opposed  to  point 
estimates  when  describing  predicted  outcomes  with  and  without 
alternatives. 

•  The  relative  uncertainty  of  alternative  plans  should  be  presented,  as 
uncertainty  in  outcomes  may  be  considered  during  plan  comparison, 
and  is  an  important  part  of  an  overall  risk  management/communica¬ 
tion  strategy. 

•  If  the  recommended  plan  has  uncertainty  that  can  be  practically 
reduced  through  post-construction  monitoring,  assessment  and 
adjustment,  an  adaptive  management  plan  should  be  developed  to 
manage  risks  and  maximize  realized  benefits. 
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Identifying  analytical  uncertainty 

All  restoration  projects  face  uncertainties,  and  identifying  the  likely  sources 
is  the  first  step  in  uncertainty  management.  Although  uncertainties  can 
arise  at  any  point  in  a  study,  the  identification,  classification,  and 
documenttation  of  uncertainties  is  critical  during  the  development  of  a 
CEM,  during  modeling  and  forecasting,  and  during  formulation  of  the 
monitoring  and  adaptive  management  plan.  Sources  of  uncertainty  can 
include  the  following: 

•  Ecosystem  uncertainty,  which  is  due  to  incomplete  description  and 
understanding  of  relevant  ecosystem  structure  and  function,  or 
unpredictable  and  highly  stochastic  events  and  interactions  affecting  key 
processes  (e.g.  flooding,  fire,  drought,  regional  climate  change,  etc.). 

•  Model  uncertainty,  which  arises  from  incomplete  knowledge,  bias,  or 
error  in  the  structure  of  a  model,  often  as  a  result  of  a  lack  of 
knowledge  or  because  of  approximations  used  to  simplify  computation. 

•  Quantity  uncertainty,  which  encompasses  the  uncertainty  in  specific 
parameters  or  input  data  used  in  a  model. 

•  Scenario  uncertainty,  which  arises  from  inaccurate  specification  of  the 
cause-and-effect  linkages  between  management  measures  and  their 
predicted  ecosystem  effects. 

•  Implementation  uncertainty,  which  is  due  to  potential  policy,  funding, 
or  other  external  factors  that  might  influence  the  timing  or  degree  of 
project  implementation. 

Quantifying  uncertainty 

The  ability  to  quantify  uncertainties  depends  upon  a  number  of  factors 
including  the  source,  available  data  and  tools,  and  the  extent  to  which  the 
associated  phenomena  are  understood.  Some  uncertainty  is  sufficiently 
tractable  that  it  can  be  described  in  terms  of  statistical  probabilities;  this 
situation  is  ideal  because  it  facilitates  easier  risk  assessment  and  decision¬ 
making.  Frequency  distributions,  statistical  variances,  coefficients  of 
variation,  confidence  intervals,  and  probability  distributions  are  commonly 
used  to  describe  the  uncertainty  in  quantities.  Of  these,  probability 
distributions  offer  the  most  complete  and  compact  form  of  representation. 
The  emerging  approach  for  these  situations  is  essentially  probabilistic 
rather  than  deterministic  (Landres  et  al.  1999). 
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Some  uncertainties,  while  identifiable  and  describable,  are  difficult  to 
objectively  quantity.  A  Likert  Scale  can  be  employed  to  allow  subject 
matter  experts  to  express  their  certainty  and  uncertainty  in  these 
situations.  Table  7  reproduces  a  Likert  Scale  used  by  authors  of  the 
Intergovernmental  Panel  on  Climate  Change  (IPCC)  Third  Assessment 
Report  to  represent  their  judgment  in  the  validity  of  a  conclusion  in  the 
report  (IPCC  2001,  p.  44).  Another  simple  form  of  expressing  uncertainty 
is  accomplished  by  giving  the  mean  plus  or  minus  some  limits  (often  a 
standard  deviation  or  range).  When  assessing  uncertainty  in  a  more 
complete  manner,  the  nature  of  the  statistical  distribution  that  is 
appropriate  for  a  given  situation  needs  to  be  defined. 


Table  7.  Verbal  and  mathematical  expression  of  certainties  and  probabilities 

(from  IPCC  2001,  p.  44). 


Verbal  Expression  of  Certainty  (indicating 
likelihood) 

Corresponding  Mathematical 
Expression  of  Probability 

Virtually  certain 

0.99  to  1.00 

Very  likely 

0.90  to  0.99 

Likely 

0.66  to  0.90 

Medium  likelihood 

0.33  to  0.66 

Unlikely 

0.10  to  0.33 

Very  unlikely 

0.01  to  0.10 

Exceptionally  unlikely 

0.00  to  0.01 

A  variety  of  tools  and  techniques  exist  for  assessing  and  quantifying 
uncertainty  in  estimating  ecosystem  benefits  (see  Convertino  et  al.  (2012) 
for  details).  Casper  et  al.  (2010)  present  an  approach  to  documenting  and 
rating  uncertainties  in  the  initial  development  of  CEMs.  Yoe  et  al.  (2010) 
discuss  a  diverse  array  of  approaches  to  evaluating  uncertainty  in  benefits 
estimates,  including  qualitative  narratives,  model  sensitivity  analysis,  use 
of  scenarios,  and  probabilistic  methods.  Fischenich  (2011)  discusses 
uncertainty  in  stream  restoration  projects,  while  McKay  and  Fischenich 
(in  preparation)  describe  the  application  of  a  Monte  Carlo  analysis  to 
evaluating  quantity  and  scenario  uncertainty  in  estimated  benefits  for  a 
coastal  restoration  project.  It  is  possible  to  assign  probabilities  to 
scenarios  as  well.  Although  it  can  sometimes  be  difficult  to  create  such 
estimates,  techniques  are  available  that  can  be  used  to  overcome  such 
difficulties  (Vose  2000). 
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Example:  Characterizing  and  Quantifying  Uncertainty 

A  habitat-suitable  model  for  a  wetlands  restoration  utilizes  three  variables:  Mi  -  Percent  of  area 
covered  by  emergent  vegetation,  M2  -  Percent  of  vegetation  on  preferred  species  list,  and  M3  - 

Average  annual  salinity.  Suitability  curves  are  as  shown  below,  and  the  composite  suitability 

1 

index  is  determined  by:  SI  =  (l^  +  V2  +  V3)3.  The  table  below  shows  a  typical  assessment  of  a 
50-acre  wetland  construction  in  an  estuary  with  200  acres  of  existing  marsh.  For  this  simple 
deterministic  analysis,  the  resulting  net  gain  in  AAHUs  is  51,  based  upon  the  best  projections  of 
future  condition  and  assumptions  regarding  the  effects  of  the  restoration  and  outside  stressors. 
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FWOP 

Variable 

TY  0 

TY  1 

TY  10 

TY  50 

Value 

SI 

Value 

SI 

Value 

SI 

Value 

SI 

VI 

60 

1.00 

60 

1.00 

55 

1.00 

40 

0.82 

V2 

50 

0.55 

50 

0.55 

50 

0.55 

40 

0.46 

V3 

10 

1.00 

10 

1.00 

10 

1.00 

15 

0.50 

Acreage 

200 

199 

190 

156 

HSI  = 

0.82 

0.82 

0.82 

0.57 

HU  = 

163.9 

163.1 

155.7 

89.5 

CUM  = 

163.5 

1434.5 

4848.6 

AAHUs  128.9 

Future  With  Project 

Variable 

TY  0 

TY  1 

TY  10 

TY  50 

Value 

SI 

Value 

SI 

Value 

SI 

Value 

SI 

VI 

60 

1.00 

70 

1.00 

65 

1.00 

50 

1.00 

V2 

50 

0.55 

65 

0.69 

65 

0.69 

65 

0.69 

V3 

10 

1.00 

10 

1.00 

10 

1.00 

15 

0.50 

Acreage 

200 

249 

238 

195 

HSI  = 

0.82 

0.88 

0.88 

0.70 

HI  = 

163.9 

219.5 

209.8 

136.5 

CUM  = 

191.2 

1932.1 

6874.1 

AAHUs 

179.9 

NET  CHANGE  IN  AAHUs  DUE  TO  PROJECT  51.0 

Sources  of  Uncertainty 

Each  of  the  sources  of  uncertainty  described  above  exists  for  the  example  project,  and  could  be 
addressed  in  the  model.  For  this  example,  the  following  two  sources  are  incorporated: 

Ecosystem  uncertainty,  due  to  unpredictable  and  highly  stochastic  effects  of  hurricanes  and 
global  and  regional  climate  change,  could  affect  the  predicted  values  for  wetland  area.  This 
could  be  addressed  using  scenario  analyses,  employing  higher  and  lower  estimates  of  marsh 
area  to  effectively  bracket  the  range  of  likely  area.  For  the  assessment,  loss  rates  were  set  at 
0.25,  0.5,  and  1.0%  for  the  low,  medium,  and  high  loss  rates.  The  resulting  net  AAFlUs  are  54.2 
for  the  low  loss  rate  and  41.5  for  the  high  loss  rate,  which  can  be  presented  as  bracketing  the 
best  estimate  of  51.0  AAFlUs  (see  tables  below). 

FWOP  -  Low  SLR 

FWOP  High  SLR 

TY  0 

TY  1 

TY  10 

TY  50 

TY  0 

TY  1 

TY  10 

TY  50 

Variable 

Value 

SI 

Value 

SI 

Value 

SI 

Value 

SI 

Variable 

Value 

SI 

Value 

SI 

Value 

SI 

Value 

SI 

VI 

60 

1.00 

60 

1.00 

55 

1.00 

40 

0.82 

VI 

60 

1.00 

60 

1.00 

55 

1.00 

40 

0.82 

V2 

50 

0.55 

50 

0.55 

50 

0.55 

40 

0.46 

V2 

50 

0.55 

50 

0.55 

50 

0.55 

40 

0.46 

V3 

10 

1.00 

10 

1.00 

10 

1.00 

15 

0.50 

V3 

10 

1.00 

10 

1.00 

10 

1.00 

15 

0.50 

Acreage 

200 

200 

195 

176 

Acreage 

200 

198 

191 

121 

HSI  = 

0.82 

0.82 

0.82 

0.57 

HSI  = 

0.82 

0.82 

0.82 

0.57 

HU  = 

163.9 

163.9 

159.8 

101.0 

HU  = 

163.9 

162.3 

156.5 

69.4 

CUM  = 

163.9 

1456.6 

5184.6 

CUM  = 

163.1 

1434.5 

4404.4 

AAHUs  136.1 

AAHUs  120.0 

Future  With  Project  -  Low  SLR 

Future  With  Project  -  High  SLR 

TY  0 

TY  1 

TY  10 

TY  50 

TY  0 

TY  1 

TY  10 

TY  50 

Variable 

Value 

SI 

Value 

SI 

Value 

SI 

Value 

SI 

Variable 

Value 

SI 

Value 

SI 

Value 

SI 

Value 

SI 

VI 

60 

1.00 

70 

1.00 

65 

1.00 

50 

1.00 

VI 

60 

1.00 

70 

1.00 

65 

1.00 

50 

1.00 

V2 

50 

0.55 

65 

0.69 

65 

0.69 

65 

0.69 

V2 

50 

0.55 

65 

0.69 

65 

0.69 

65 

0.69 

V3 

10 

1.00 

10 

1.00 

10 

1.00 

15 

0.50 

V3 

10 

1.00 

10 

1.00 

10 

1.00 

15 

0.50 

Acreage 

200 

249 

244 

221 

Acreage 

200 

248 

226 

151 

HSI  = 

0.82 

0.88 

0.88 

0.70 

HSI  = 

0.82 

0.88 

0.88 

0.70 

HU  = 

163.9 

219.5 

215.1 

154.7 

HU  = 

163.9 

218.6 

199.2 

105.7 

CUM  = 

191.2 

1955.9 

7368.1 

CUM  = 

190.8 

1880.5 

6007.8 

AAHUs  190.3 

AAHUs  161.6 

NET  CHANGE  IN  AAHUs  DUE  TO  PROJECT  54.2 

NET  CHANGE  IN  AAHUs  DUE  TO  PROJECT  41.5 

Quantity  uncertainty,  which  encompasses  the  uncertainty  in  estimates  of  future  conditions  for 
the  variables  used  in  a  model.  For  example,  confidence  may  be  low  that  the  percent  vegetation 
cover  in  year  50  for  the  FWOP  will  be  exactly  40,  and  it  is  likely  the  value  would  be  somewhat 
higher  or  lower  than  this  estimate.  Depending  upon  the  sensitivity  of  the  model  to  this  variable 
and  the  modeler’s  confidence  in  alternative  values,  the  models  can  be  run  with  a  range  of 
values  of  equal  probability  (eg.  30  to  50  percent),  or  as  some  probability  distribution  (eg.  40 
+/~14.5%  ).  If  several  model  variables  have  ranges  of  values,  a  Monte  Carlo  analysis  might  be 
needed  to  assess  a  set  of  possible  combinations  of  values.  A  Monte  Carlo  analysis  employing 
ranges  of  values  for  all  three  variables  as  well  as  uncertain  sea  level  rise  effects  with  1000 
iterations  yielded  a  mean  net  AAFIU  of  50.8,  with  95%  confidence  limits  of  40.3  and  55.1 
(calculations  not  shown). 

ER  DC/EL  TR-13-4 


36 


Uncertainty  and  risk  management 

Despite  a  vast  academic  and  professional  literature  addressing  many 
aspects  of  risk  management,  implementing  these  concepts  for  ecosystem 
restoration  planning  is  a  far  from  mainstream  effort.  Risk,  over  a  given 
time,  is  a  product  of  likelihoods  and  consequences  of  adverse  outcomes. 

This  definition  implies  that  four  aspects  are  involved  in  considering  risk— a 
time  scale,  scenarios,  relevant  consequences,  and  corresponding  likelihoods 
or  probabilities  (Beer  2006).  In  a  typical  risk  assessment,  the  following 
questions  are  addressed  as  part  of  the  overall  risk  management  process 
(after  Suedel  et  al.  (2012)): 

1.  What  can  go  wrong? 

2.  What  is  the  likelihood  that  it  will  go  wrong? 

3.  What  are  the  consequences? 

4.  What  can  be  done  to  mitigate  the  risks? 

A  corresponding  risk  management  framework  for  ecosystem  restoration 
projects  would  involve  subjecting  each  alternative  and  the  FWOP  to  the 
above  considerations.  In  cases  where  the  potential  regret  from  a  “risky” 
alternative  is  high,  other  alternatives  may  be  favored  or  the  risks  can  be 
reduced  to  acceptable  levels.  Risk  reduction  strategies  for  ecosystem 
restoration  projects  usually  involve  adaptive  management,  but  might  also 
include  more  detailed  analyses  that  reduce  uncertainties  or  reformulation/ 
redesign  of  management  measures  to  make  them  more  resilient  or  adaptive 
under  scenarios  likely  to  affect  performance. 

Reference  concepts 

Reference  concepts  have  long  been  used  to  support  ecosystem  restoration. 
The  notion  of  a  reference  system  that  serves  both  as  a  “guiding  image”  for 
what  restoration  should  look  like  and  a  benchmark  for  assessing  specific 
details  of  the  restoration  is  appealing.  Consequently,  ecosystem  references 
have  been  used  to  set  restoration  or  mitigation  priorities,  develop  ecosystem 
restoration  designs,  support  ecological  monitoring  programs,  evaluate 
sustainability,  and  set  and  assess  restoration  success  criteria,  among  other 
uses.  Miller  et  al.  (2012)  provide  an  overview  of  the  different  ways  in  which 
reference  concepts  have  been  used  and  discuss  the  strengths  and  limitations 
of  alternative  reference  formulations  (see  Table  8). 
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Table  8.  Approaches  to  characterizing  reference  condition  for  use  in  ecosystem  restoration  project  planning 

(adapted  from  Miller  et  al.  2012). 


Reference 

Approach 

Description 

Applicable 

Reference 

Condition1 

Requirements/ 

Assumptions 

Benefits 

Limitations 

On-site  analogous 

A  present,  on-site 
condition  (within 
project  footprint) 

LDC  most  likely 

Requires  enough  on-site 

information  to  determine 

degree  of  function  and 
degradation  and  to  set 
targets;  may  require 

consideration  of  broader 

watershed  conditions 

Low  mobilization 

costs,  parallel 
stressors,  many 
parameters  equal 
(e.g.,  hydrology) 

May  not  represent 
target  reference 
condition,  may  not 
represent  range  of 
condition,  may  not 
represent  ecological 
trajectory 

Off-site  analogous 

A  present, 

ecologically 

representative 

condition  outside 

project  footprint 

LDC  most  likely, 

MDC  possible 

Requires  enough 

information  at  a  suitable 

off-site  location  to 

determine  degradation 
and  set  target, 
comparable  class  of 
system  with  parallel 
stressors,  measurable 
P/C/B  parameters 

More  likely  to  find 

reference  that  can 

help  define  target 

reference  condition 

with  parallel 
stressors,  parameters 

May  not  represent 
range  of  condition,  may 
not  represent  target 
reference  condition, 

more  cost  to  locate  and 

characterize  another 

site,  may  not  represent 
ecological  trajectory 

Historical 

reference 

A  selected  historical 

condition  within 

project  area  (can  be 
applied  as  off-site 
analogous  approach 

if  conditions  are 

met) 

HCPI  or  HCPA 

HCPD  if  a  specific 

isolated  event 

caused 

disturbance 

Requires  the  right  data 
type  /  resolution  to  set 
targets  matched  to 
objectives;  if  on-site,  may 
not  require  classification 

Opportunity  to 
characterize 

adjustment  of 
processes  to  known 
stressors,  if  stressors 
not  in  flux  or  pre¬ 
disturbance  data  are 

proximal,  can 
represent  target 

reference  condition  or 

MDC 

Stressors  may  have 
changed,  other 
parameters  may  be 
changing,  constraints 
may  eliminate  historical 

reference  from 

consideration  as  target 
reference  condition 

Virtual  (also  called 
constructed ) 

Developed  from  a 
combination  of 

sources  to 

represent  target 

reference  condition 

for  given  physical 
setting,  other 

constraints 

Any  of  HCPI,  HCPA, 
HCPD,  MDC  or  LDC 

in  combination  with 

site  or  other  data 

Typically  requires  data 
from  multiple  sources, 

Best  Professional 

Judgement  (BPJ),  and 

models 

Highly  flexible  if  good 
information  is 

available,  high 
resolution  in  defining 
target  condition,  best 
for  use  in  settings  with 
many  constraints, 

costs  can  be  low  if 

existing  models,  BPJ, 

and  collaborative 

processes  are  used 

Highly  dependent  on 
good  information,  good 
interpretation/ 
analysis  of  available 
information,  can  be 
quite  reliant  on  models, 
and  subject  to  notable 
debate,  costs  can  be 
high  if  requiring  new  or 
extensive  modeling, 

Regional  Index 

A  range  of  existing 

reference  sites 

reflecting  a 

continuum  of 

conditions 

MDC 

Requires  classification 

and  considerable  data  to 

characterize  the  range  of 

conditions  to  evaluate 

degradation  and  set 
targets 

Highly  robust 
representative  of  full 
range  of  conditions, 
puts  projects  into 
context,  best 

characterization  of 
target  reference 

condition 

Highly  data  dependent, 
can  take  years  to 
develop,  and  can  be 
costly 

1  BAC  -  Best  Attainable  Condition;  LDC  -  Least  Disturbed  Condition;  MDC  -  Minimally  Disturbed  Condition;  HC  -  Historic  Condition 
(PD-  Pre-Disturbed;  PI-  Pre-Industrial;  PD-  Pre-Disturbance) 
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A  key  application  of  reference  concepts  for  EBA  is  the  use  of  a  reference  as  a 
means  of  scaling  ecological  benefits;  indexing  the  degree  to  which  a  project 
ecosystem  achieves  the  structure  and  function  of  a  reference  is  a  way  of 
ascribing  "value"  to  an  ecologically-motivated  project.  In  a  simple  sense,  a 
Reference  Condition  Index  (RCI)  can  be  established  using  any  number  of 
ecosystem  attributes  and  would  replace  the  “habitat  quality  index” 
commonly  used  as  part  of  a  HEP  analysis.  Pruitt  et  al.  (2012)  provide  an 
overview  of  the  ways  in  which  reference  concepts  can  be  applied  to  an  EBA 
and  more  detail  on  the  application  of  an  RCI.  Figure  6  below  is  a  schematic 
representation  of  an  application  adapted  from  Pruitt  et  al.  (2012),  and  can 
be  loosely  interpreted  as  showing  an  ecosystem  in  a  current  state  at  60%  of 
full  function,  with  alternative  future  states  of  30%  (FWOP)  and  75%  (with- 
project).  The  index  values  for  the  various  states  can  be  multiplied  by  the 
number  of  acres  to  obtain  an  output  analogous  to  habitat  units. 


Figure  6.  Schematic  representation  of  a  Reference  Condition  Index  (RCI)  for  assessing 
alternative  benefits.  The  RCI  value  for  the  historic  condition  is  1.0,  for  the  current  condition  is 
0.6,  for  the  FWOP  is  0.3,  and  for  the  alternative  achieving  the  BAC  is  0.75. 


Adaptive  management  (AM) 

As  implied  by  the  term,  “adaptive  management”  prescribes  a  process 
wherein  management  actions  can  be  changed  in  relation  to  their  efficacy 
in  restoring  and/or  maintaining  an  ecological  or  engineered  system  in  a 
specified  desired  state  (Walters  1986).  The  desired  state  (e.g.,  goals  and 
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objectives)  might  be  some  precisely  defined  structural  condition  or,  more 
realistically,  a  range  of  structural  conditions,  rates  of  ecological  processes, 
or  some  description  of  biotic  potential  (e.g.,  productivity).  Adaptive 
management  helps  to  achieve  desired  goals  by  addressing  uncertainty, 
incorporating  flexibility  and  robustness  into  project  design,  and  using  new 
information  to  inform  decision-making. 

Role  of  adaptive  management  in  EBA 

The  development  of  an  adaptive  management  plan  is  now  required  for  all 
ecosystem  restoration  feasibility  studies.1  Adaptive  management  provides  a 
decision-making  framework  that  can  adjust  management  actions  based  on 
newly  acquired  information  and  monitored  outcomes  of  previous  decisions. 
Importantly,  this  adaptive  decision-making  process  can  increase  the 
chances  that  management  goals  and  objectives  (e.g.,  ecosystem  restoration 
or  sustainability)  will  be  achieved  despite  uncertainties.  While  adaptive 
management  has  traditionally  been  viewed  as  a  post-project  implementa¬ 
tion  activity,  the  planning  phase  for  adaptive  management  is  critical  to  its 
successful  implementation.  The  nature  of  AM  planning  is  such  that  it  allows 
for  adjustments  to  plans  that  result  in  increased  benefits.  Implementation 
of  AM  measures  improves  the  realization  of  benefits,  and  associated 
monitoring  contributes  to  the  accounting  of  project  benefits. 

Adaptive  management  planning 

"Adaptive  management  (AM)  does  not  represent  an  end  in  itself,  but  rather 
a  means  to  more  effective  decisions  and  enhanced  benefits.  Its  true  measure 
is  in  how  well  it  helps  meet  environmental,  social,  and  economic  goals, 
increases  scientific  knowledge,  and  reduces  tensions  among  stakeholders” 
(NRC  2004).  AM  can  be  regarded  in  terms  of  two  principle  phases;  the 
planning  phase  wherein  the  proposed  adaptive  management  strategy  is 
developed,  and  an  implementation  phase  during  which  it  is  executed.  The 
planning  phase  is  summarized  below  while  the  implementation  phase  is 
summarized  in  the  section  titled  “Monitoring  and  adaptive  management 
(implementation).”  Fischenich  et  al.  (2012)  and  Barr  et.  al.  (in  preparation) 
provide  more  details  on  the  application  of  adaptive  management. 


1  On  August  31,  2009,  a  memorandum  from  CECW-PB,  was  issued  to  Commanders  and  Major  Subordinate 
Commands  providing  detailed  requirements  for  implementation  of  Section  2039  of  WRDA  2007.  http://cw- 
environment.usace.anny.mil/pdfs/09sep2-wrda-monitor.pdf 
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Adaptive  management  adds  several  considerations  to  the  traditional 
planning  process.  The  identification  and  assessment  of  performance 
measures  and  decision  criteria,  as  well  as  required  monitoring  before, 
during,  and  following  project  construction  are  obvious  needs.  Adaptive 
management  planning  also  requires  consideration  of  the  flexibility  and 
reversibility  of  alternatives,  and  a  determination  of  what  adjustments  to 
the  project  restoration  actions  may  be  needed.  Plans  must  also  be  made 
for  the  acquisition  and  management  of  data,  as  well  as  the  analysis  and 
decision-making  required  to  implement  management  decisions.  These 
requirements  force  planning  teams  to  contemplate  objectives  and  project 
performance  at  a  high  level  of  detail,  and  the  resulting  plans  inevitably 
benefit  from  this  additional  degree  of  thought. 

The  planning  phase  for  AM  is  iterative  in  nature  and  comes  to  play  in  all  six 
steps  of  the  traditional  planning  process.  Planning  for  AM  means  carefully 
assessing  l)  critical  uncertainties  associated  with  project  execution,  2)  the 
potential  to  address  those  uncertainties  through  systematic  monitoring  and 
adjustment  of  project  features,  and  3)  an  assessment  of  the  costs  and  poten¬ 
tial  benefits  from  implementation  of  an  AM  plan.  This  process  requires 
planners  to  carefully  consider  potential  outcomes  for  each  alternative, 
including  various  outcomes  associated  with  key  uncertainties,  and  to 
contemplate  what  (if  any)  actions  might  be  taken  should  the  outcome  differ 
from  that  which  is  desired.  This  can  influence  the  formulation  of  alterna¬ 
tives  as  certain  design  strategies  lend  themselves  more  readily  to  AM.  The 
explicit  treatment  of  uncertainty  as  part  of  planning  and  the  development  of 
contingency  plans  to  address  undesirable  or  unexpected  responses  are  at 
the  core  of  AM. 

Documentation 

Need  and  approach 

Documentation  is  critical  for  presenting  a  transparent  and  logical  decision¬ 
making  process  and  a  defensible  recommendation.  The  decision  document 
must  tie  together  the  CEM  and  its  linkage  to  the  planning  objectives,  evalua¬ 
tion  metrics,  model  selection,  project  benefits,  plan  selection,  remaining 
uncertainties,  and  how  uncertainties  will  be  addressed  through  adaptive 
management.  Credible  documentation  that  demonstrates  the  rationale  for 
plan  selection  is  particularly  important  for  ecosystem  restoration  projects, 
since  plan  selection  is  not  strictly  dictated  by  economic  evaluation. 
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The  process  of  conducting  an  EBAfor  an  ecosystem  restoration  project  can 
take  months  to  years,  involving  hundreds  of  decisions  by  several  indivi¬ 
duals.  Therefore,  it  is  critical  that  the  key  decisions  be  documented  as  they 
occur ;  it  is  simply  not  possible  to  recall  the  details  with  sufficient  accuracy 
to  properly  document  decisions  if  documentation  occurs  only  at  periodic 
review  stages. 

Scope  and  content 

Requirements  for  documentation  associated  with  the  planning  process  for 
the  USACE  are  spelled  out  in  the  Planning  Guidance  Notebook  (PGN)  and 
in  several  policy  documents.  With  regards  to  the  EBA  specifically,  it  is 
important  to  tell  the  story  of  how  the  study  evolved  from  an  understanding 
of  the  problem  and  opportunities  through  the  many  analyses  and  decisions 
to  the  selection  of  the  recommended  plan.  Particular  attention  should  be 
paid  to  the  following: 

•  Identification  of  the  problem  and  presentation  of  the  CEM. 

•  Complete  and  clear  statement  of  objectives. 

•  Logic  and  rationale  for  selected  metrics  and  which  objectives  they 
measure. 

•  Role  of  adaptive  management  in  shaping  alternatives,  metrics,  or 
outputs. 

•  Available  and  selected  techniques  for  assessing  and  forecasting  the 
metric  (e.g.,  numerical  models,  expert  judgment,  monitoring  plans, 
data  collection  protocols). 

•  Literature,  expert,  or  past-project  support  for  use  of  the  metrics  and 
models. 

•  Assumptions  and  limitations  associated  with  metrics,  models,  and 
analysis. 

•  Application  of  professional  judgment  in  metric  or  model  development 
or  assessment. 

•  Identification,  classification,  and  quantification  of  critical 
uncertainties. 

•  Basis  for  any  scenarios  used  in  the  analyses. 

•  Any  review  the  metric  set  or  models  have  undergone  (e.g.,  interagency 
project  team). 

•  Results  of  forecasting,  including  plots  of  outputs  over  time. 

•  Results  of  CE/ICA  analyses  including  appropriate  figures.  Decision 
criteria  applied  to  assessment  results  and  corresponding  preferred 
alternative. 


ER  DC/EL  TR-13-4 


42 


Units  and  significant  digits 

It  is  important  that  benefits  be  expressed  in  proper  units  and  at  an 
appropriate  level  of  precision.  Measurements  always  have  a  limited 
precision;  and  many  estimates  used  in  calculating  benefits  are  more 
limited  in  precision  than  the  actual  measurements.  Because  these  inputs 
have  limited  precision,  the  results  of  calculations  based  upon  them 
likewise  have  limited  precision.  Significant  figures  track  measurement 
precision.  The  rules  for  significant  figures  are  straightforward,  and  can  be 
found  in  many  texts  or  in  on-line  resources. 
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4  Evaluating  Project  Success 

Performance  measures  and  monitoring 

Decision  criteria1  are  used  to  determine  project  success  and  if  and  when 
adaptive  management  actions  should  be  implemented.  Decision  criteria 
can  be  expressed  in  terms  of  performance  measures  or  risk  endpoints. 
Performance  measures  (or  targets )  are  quantified  expressions  of  project 
objectives  at  a  particular  point  in  time.  Performance  measures  are  derived 
from  project  objectives  and  should:  l)  be  measurable;  2)  have  a  relatively 
strong  degree  of  predictability  (i.e.,  targets  specified  by  predictive  models 
or  by  best  professional  judgment);  3)  change  in  response  to  project 
implementation;  and  4)  verily  progress  and  evaluate  hypotheses  through 
monitoring  and  assessment. 

Risk  endpoints  refer  to  undesired  effects  of  management  actions;  they  are 
essentially  measures  of  negative  project  performance  (i.e.  adverse  impacts 
or  constraint  violations).  The  concept  of  risk  includes  1)  the  possibility  that 
the  anticipated  project  outcomes  will  not  be  achieved,  2)  the  potential  that 
some  other  unexpected,  undesired  (and  perhaps  irreversible)  outcome  will 
occur,  or  3)  the  knowledge  that  certain  adverse  impacts  are  to  be  avoided, 
minimized,  or  mitigated  during  or  following  project  implementation. 

The  decision  criteria  can  be  specified  as  single  values  or  ranges  of  desirable 
outcomes.  They  can  be  qualitative  or  quantitative  based  on  the  nature  of  the 
performance  measure  and  the  level  of  information  necessary  to  make  a 
decision,  but  should  be  quantified  when  possible.  Because  of  the  long 
response  time  for  many  ecosystem  restoration  efforts,  decision  criteria  are 
often  based  upon  trajectories  or  rates  of  change  for  metrics  that  are 
indicative  of  ecological  function.  Success  for  ecosystem  restoration  projects 
occurs  when  decision  criteria  suggest  that  the  objectives  have  been  met,  or 
that  specific  thresholds  have  been  crossed  and  the  ecosystem  is  on  a 
recovery  trajectory  that  will  result  in  the  achievement  of  the  objectives. 


1  Sometimes  referred  to  as  adaptive  management  triggers. 
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Monitoring  and  adaptive  management  (implementation)1 

Implementation  of  the  Adaptive  Management  Plan  is  fairly 
straightforward,  but  needs  technical,  management,  and  financial  support 
as  well  as  a  functional  governance  structure  with  sufficient  authority  to 
make  decisions.  The  implementation  steps2 3  are  as  follows: 

1.  Results  of  the  ongoing  monitoring  programs  are  collated  and  analyzed  by 
the  Adaptive  Management  Team  (AMTs)  to  assess  whether  any 
performance  measures  or  risk  endpoints  are  triggered. 

2.  If  none  of  the  decision  criteria  are  triggered,  the  adaptive  management 
process  can  simply  continue  with  the  current  monitoring  programs  until 
the  next  evaluation  is  performed. 

3.  If  decision  criteria  are  triggered,  the  AMT  evaluates  the  circumstances  and 
decides  to  a)  implement  prescribed  adjustments  to  the  management 
actions,  b)  undertake  additional  monitoring  or  study,  or  c)  redress  the 
performance  standards  or  risk  endpoints  that  have  not  been  met.  This 
approach  permits  flexibility  in  interpreting  monitoring  results  and  allows 
for  adjustments  to  the  process  and  criteria  as  warranted. 

4.  Following  resolution  of  the  AMT  recommendations  for  adjustments  to  the 
management  actions,  the  adaptive  management  process  continues  by 
cycling  back  to  step  1.  This  process  continues  until  either  a)  criteria  for 
project  success  have  been  met  as  determined  by  the  Division  Commander, 
or  b)  federal  funding  can  no  longer  be  used  for  monitoring  or  adaptive 
management  (currently  after  10  years). 

One  of  the  most  important  aspects  of  an  adaptive  management  process  is 
documentation.  Implementation  of  adaptive  management  emphasizes  an 
open  and  transparent  management  practice  wherein  the  results  of  moni¬ 
toring,  assessment,  and  decision-making  are  routinely  and  consistently 
documented.  The  set-up  phase  and  the  resulting  adaptive  management  plan 
should  specify  the  provisions  for  regularly  documenting  adaptive 
management. 


1  Implementation  Guidance  for  Section  2039  of  the  Water  Resources  Development  Act  of  2007  (WRDA 
2007)— Monitoring  Ecosystem  Restoration,  Memorandum  from  the  USACE  Chief  of  Planning  dated  31 
August  2009.  http://cw-environment.usace.armv.mil/pdfs/09sep2-wrda-monitor.pdf 

2  Policies  regarding  monitoring  and  adaptive  management  are  evolving.  Planners  are  encouraged  to 
familiarize  themselves  with  current  policy  and  note  that  the  approach  described  herein  must  be 
adjusted  as  needed  to  conform  to  current  policy  requirements. 

3  The  composition  and  role  of  the  AMT  is  determined  during  the  AM  planning  phase  and  can  include 
representatives  from  the  USACE,  key  resource  agencies,  and/or  the  stakeholders. 
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5  Supporting  Information  and  Guidance 

EBA  Program  products 

The  work  presented  in  this  technical  report  was  conducted  by  the 
U.S.  Army  Engineer  Research  and  Development  Center  (ERDC)  as  part  of 
the  EBA  Program.  Numerous  additional  products  have  been  prepared 
under  the  program,  including  technical  notes  and  reports,  journal  papers, 
models,  tools,  fact  sheets,  and  webinars.  These  products  provide  more 
detailed  information  on  many  of  the  topics  presented  in  this  report,  and 
can  be  accessed  through  the  Ecosystem  Restoration  Gateway  (discussed 
below),  or  directly  from  the  EBA  Program  website  at: 

http://el.erdc.usace.armv.mil/prcgrams.cfm?Tppic=eba&Qpticn=Prcgram 

Ecosystem  Restoration  Gateway 

The  Civil  Works  Environment  Gateway  was  established  as  the  Corps'  web 
portal  to  news,  information,  guidance,  tools,  data,  and  other  resources 
supporting  the  community  of  practice  within  the  environmental  business 
line.  Information  about  environmental  benefits  assessment  within  the 
Corps  and  the  products  of  this  research  effort  are  accessible  through  the 
Gateway  at: 


http://cw-envirpnment.usace.armv.mil/cwenv.cfm 
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